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Introduction 


■ And it ought to be remembered that there is nothing more 

difficult to take in hand, more perilous to conduct, or 
more uncertain in its success, than to take the lead in the 
introduction of a new order of things. 

— N. Machiavelli, 1513 

Elements of Programming Interviews (EPI) aims to help engineers interviewing for 
software development positions. The primary focus of EPI is data structures, al¬ 
gorithms, system design, and problem solving. The material is largely presented 
through questions. 

An interview problem 

Let's begin with Figure 1 below. It depicts movements in the share price of a company 
over 40 days. Specifically, for each day, the chart shows the daily high and low, and 
the price at the opening bell (denoted by the white square). Suppose you were asked 
in an interview to design an algorithm that determines the maximum profit that could 
have been made by buying and then selling a single share over a given day range, 
subject to the constraint that the buy and the sell have to take place at the start of the 
day. (This algorithm may be needed to backtest a trading strategy.) 

You may want to stop reading now, and attempt this problem on your own. 

First clarify the problem. For example, you should ask for the input format. 
Let's say the input consists of three arrays L, H, and S, of nonnegative floating point 
numbers, representing the low, high, and starting prices for each day. The constraint 
that the purchase and sale have to take place at the start of the day means that it 
suffices to consider S. You may be tempted to simply return the difference of the 



Day 0 Day 5 Day 10 Day 15 Day 20 Day 25 Day 30 Day 35 Day 40 


Figure 1: Share price as a function of time. 
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minimum and maximum elements in S. If you try a few test cases, you will see that 
the minimum can occur after the maximum, which violates the requirement in the 
problem statement—you have to buy before you can sell. 

At this point, a brute-force algorithm would be appropriate. For each pair of 
indices i and j > i, if S[j] - S[z] is greater than the largest difference seen so far, 
update the largest difference to S[j] - S[z]. You should be able to code this algorithm 
using a pair of nested for-loops and test it in a matter of a few minutes. You should 
also derive its time complexity as a function of the length n of the input array. The 
outer loop is invoked n - 1 times, and the ith iteration processes n - 1 - i elements. 
Processing an element entails computing a difference, performing a compare, and 
possibly updating a variable, all of which take constant time. Hence, the run time is 
proportional to 0 2 (n - 1 — i) = i.e., the time complexity of the brute-force 

algorithm is 0(n 2 ). You should also consider the space complexity, i.e., how much 
memory your algorithm uses. The array itself takes memory proportional to n, and 
the additional memory used by the brute-force algorithm is a constant independent 
of n —a couple of iterators and one floating point variable. 

Once you have a working algorithm, try to improve upon it. Specifically, an 
0(n 2 ) algorithm is usually not acceptable when faced with large arrays. You may 
have heard of an algorithm design pattern called divide-and-conquer. It yields the 
following algorithm for this problem. Split S into two subarrays, S[0 : |_f J] and 
S[L|J + 1 : n - 1]; compute the best result for the first and second subarrays; and 
combine these results. In the combine step we take the better of the results for the two 
subarrays. However, we also need to consider the case where the optimum buy and 
sell take place in separate subarrays. When this is the case, the buy must be in the 
first subarray, and the sell in the second subarray, since the buy must happen before 
the sell. If the optimum buy and sell are in different subarrays, the optimum buy 
price is the minimum price in the first subarray, and the optimum sell price is in the 
maximum price in the second subarray. We can compute these prices in 0(n) time 
with a single pass over each subarray. Therefore, the time complexity T(n) for the 
divide-and-conquer algorithm satisfies the recurrence relation T(n) = 2T(|) + 0(n), 
which solves to 0(n log n). 

The divide-and-conquer algorithm is elegant and fast. Its implementation entails 
some corner cases, e.g., an empty subarray, subarrays of length one, and an array in 
which the price decreases monotonically, but it can still be written and tested by a 
good developer in 20-30 minutes. 

Looking carefully at the combine step of the divide-and-conquer algorithm, you 
may have a flash of insight. Specifically, you may notice that the maximum profit 
that can be made by selling on a specific day is determined by the minimum of the 
stock prices over the previous days. Since the maximum profit corresponds to selling 
on some day, the following algorithm correctly computes the maximum profit. Iterate 
through S, keeping track of the minimum element m seen thus far. If the difference 
of the current element and m is greater than the maximum profit recorded so far, 
update the maximum profit. This algorithm performs a constant amount of work per 
array element, leading to an 0(n) time complexity. It uses two float-valued variables 
(the minimum element and the maximum profit recorded so far) and an iterator, i.e.. 
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0(1) additional space. It is considerably simpler to implement than the divide-and- 
conquer algorithm—a few minutes should suffice to write and test it. Working code 
is presented in Solution 6.6 on Page 70. 

If in a 45-60 minutes interview, you can develop the algorithm described above, 
implement and test it, and analyze its complexity, you would have had a very suc¬ 
cessful interview. In particular, you would have demonstrated to your interviewer 
that you possess several key skills: 

- The ability to rigorously formulate real-world problems. 

- The skills to solve problems and design algorithms. 

- The tools to go from an algorithm to a tested program. 

- The analytical techniques required to determine the computational complexity 
of your solution. 

Book organization 

Interviewing successfully is about more than being able to intelligently select data 
structures and design algorithms quickly. For example, you also need to know how 
to identify suitable companies, pitch yourself, ask for help when you are stuck on an 
interview problem, and convey your enthusiasm. These aspects of interviewing are 
the subject of Chapters 1-3, and are summarized in Table 1.1 on Page 8. 

Chapter 1 is specifically concerned with preparation. Chapter 2 discusses how 
you should conduct yourself at the interview itself. Chapter 3 describes interviewing 
from the interviewer's perspective. The latter is important for candidates too, because 
of the insights it offers into the decision making process. Chapter 4 reviews problem 
solving. 

Since not everyone will have the time to work through EPI in its entirety, we have 
prepared a study guide (Table 1.2 on Page 9) to problems you should solve, based on 
the amount of time you have available. 

The problem chapters are organized as follows. Chapters 5-15 are concerned with 
basic data structures, such as arrays and binary search trees, and basic algorithms, 
such as binary search and quicksort. In our experience, this is the material that 
most interview questions are based on. Chapters 16-19 cover advanced algorithm 
design principles, such as dynamic programming and heuristics, as well as graphs. 
Chapter 20 focuses on parallel programming. 

Each chapter begins with an introduction followed by problems. The introduc¬ 
tion itself consists of a brief review of basic concepts and terminology, followed by 
a boot camp. Each boot camp is (1.) a straightforward, illustrative example that il¬ 
lustrates the essence of the chapter without being too challenging; and (2.) top tips 
for the subject matter, presented in tabular format. For chapters where the pro¬ 
gramming language includes features that are relevant, we present these features 
in list form. This list is ordered with basic usage coming first, followed by subtler 
aspects. Basic usage is demonstrated using methods calls with concrete arguments, 
e.g.. Arrays. asLi st (1 y 2,4,8). Subtler aspects of the library, such as ways to reduce 
code length, underappreciated features, and potential pitfalls, appear later in the list. 
Broadly speaking, the problems are ordered by subtopic, with more commonly asked 
problems appearing first. Chapter 25 consists of a collection of more challenging 
problems. 
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Domain-specific knowledge is covered in Chapters 21,22,23, and 24, which are 
concerned with system design, programming language concepts, object-oriented pro¬ 
gramming, and commonly used tools. Keep in mind that some companies do not 
ask these questions—you should investigate the topics asked by companies you are 
interviewing at before investing too much time in them. These problems are more 
likely to be asked of architects, senior developers and specialists. 

The notation, specifically the symbols we use for describing algorithms, e.g., 
L"To 1 * 2 , k b )r <2,3,5, 7),A[i : j], \x\, (1011)2, n\, [x\x 2 > 2), etc., is summarized start¬ 
ing on Page 514. It should be familiar to anyone with a technical undergraduate 
degree, but we still request you to review it carefully before getting into the book, 
and whenever you have doubts about the meaning of a symbol. Terms, e.g., BFS and 
dequeue, are indexed starting on Page 516. 

The EPI editorial style 

Solutions are based on basic concepts, such as arrays, hash tables, and binary search, 
used in clever ways. Some solutions use relatively advanced machinery, e.g., Dijk- 
stra's shortest path algorithm. You will encounter such problems in an interview only 
if you have a graduate degree or claim specialized knowledge. 

Most solutions include code snippets. Please read Section 1 on Page 11 to famil¬ 
iarize yourself with the Java constructs and practices used in this book. Source code, 
which includes randomized and directed test cases, can be found at the book website. 
Domain specific problems are conceptual and not meant to be coded; a few algorithm 
design problems are also in this spirit. 

One of our key design goals for EPI was to make learning easier by establishing a 
uniform way in which to describe problems and solutions. We refer to our exposition 
style as the EPI Editorial Style. 

Problems are specified as follows: 

(1.) We establish context, e.g., a real-world scenario, an example, etc. 

(2.) We state the problem to be solved. Unlike a textbook, but as is true for an 
interview, we do not give formal specifications, e.g., we do not specify the 
detailed input format or say what to do on illegal inputs. As a general rule, 
avoid writing code that parses input. See Page 15 for an elaboration. 

(3.) We give a short hint —you should read this only if you get stuck. (The hint is 
similar to what an interviewer will give you if you do not make progress.) 
Solutions are developed as follows: 

(1.) We begin a simple brute-force solution. 

(2.) We then analyze the brute-force approach and try to get intuition for why it is 
inefficient and where we can improve upon it, possibly by looking at concrete 
examples, related algorithms, etc. 

(3.) Based on these insights, we develop a more efficient algorithm, and describe it 
in prose. 

(4.) We apply the program to a concrete input. 

(5.) We give code for the key steps. 

(6.) We analyze time and space complexity. 

(7.) We outline variants —problems whose formulation or solution is similar to the 
solved problem. Use variants for practice, and to test your understanding of 
the solution. 
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Note that exceptions exists to this style—for example a brute-force solution may 
not be meaningful, e.g., if it entails enumerating all double-precision floating point 
numbers in some range. For the chapters at the end of the book, which correspond 
to more advanced topics, such as Dynamic Programming, and Graph Algorithms, 
we use more parsimonious presentations, e.g., we forgo examples of applying the 
derived algorithm to a concrete example. 

Level and prerequisites 

We expect readers to be familiar with data structures and algorithms taught at the 
undergraduate level. The chapters on concurrency and system design require knowl¬ 
edge of locks, distributed systems, operating systems (OS), and insight into commonly 
used applications. Some of the material in the later chapters, specifically dynamic 
programming, graphs, and greedy algorithms, is more advanced and geared towards 
candidates with graduate degrees or specialized knowledge. 

The review at the start of each chapter is not meant to be comprehensive and if 
you are not familiar with the material, you should first study it in an algorithms 
textbook. There are dozens of such texts and our preference is to master one or two 
good books rather than superficially sample many. Algorithms by Dasgupta, et al. is 
succinct and beautifully written; Introduction to Algorithms by Cormen, et al. is an 
amazing reference. 

Reader engagement 

Many of the best ideas in EPI came from readers like you. The study guide, ninja 
notation, and hints, are a few examples of many improvements that were brought 
about by our readers. The companion website, elementsofprogramminginterviews.com, 
includes a Stack Overflow-style discussion forum, and links to our social media 
presence. It also has links blog postings, code, and bug reports. You can always 
communicate with us directly—our contact information is on the website. 
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Part I 

The Interview 



Chapter 

l Getting Ready 


Before everything else, getting ready is the secret of success. 

— H. Ford 


The most important part of interview preparation is knowing the material and prac¬ 
ticing problem solving. However, the nontechnical aspects of interviewing are also 
very important, and often overlooked. Chapters 1-3 are concerned with the non¬ 
technical aspects of interviewing, ranging from resume preparation to how hiring 
decisions are made. These aspects of interviewing are summarized in Table 1.1 on 
the following page 

Study guide 

Ideally, you would prepare for an interview by solving all the problems in EPI. This 
is doable over 12 months if you solve a problem a day, where solving entails writing 
a program and getting it to work on some test cases. 

Since different candidates have different time constraints, we have outlined several 
study scenarios, and recommended a subset of problems for each scenario. This 
information is summarized in Table 1.2 on Page 9. The preparation scenarios we 
consider are Hackathon (a weekend entirely devoted to preparation), finals cram 
(one week, 3-A hours per day), term project (four weeks, 1.5-2.5 hours per day), and 
algorithms class (3—4 months, 1 hour per day). 

The problems in EPI are meant to be representative of the problems you will 
encounter in an interview. If you need a data structure and algorithms refresher, take 
a look at the EPI website, which includes a collection of review problems that will get 
you ready for EPI more quickly that a textbook would. 

A large majority of the interview questions at Google, Amazon, Microsoft, and 
similar companies are drawn from the topics in Chapters 5-15. Exercise common 
sense when using Table 1.2, e.g., if you are interviewing for a position with a financial 
firm, do more problems related to probability. 

Although an interviewer may occasionally ask a question directly from EPI, you 
should not base your preparation on memorizing solutions. Rote learning will likely 
lead to your giving a perfect solution to the wrong problem. 

Chapter 25 contains a diverse collection of challenging questions. Use them to 
hone your problem solving skills, but go to them only after you have made major 
inroads into the earlier chapters. If you have a graduate degree, or claim specialized 
knowledge, you should definitely solve some problems from Chapter 25. 
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Table 1.1: A summary of nontechnical aspects of interviewing 


The Interview Lifecycle, on the current 
page 

• Identify companies, contacts 

• Resume preparation 
o Basic principles 

o Website with links to projects 
o Linkedln profile & recommendations 

• Resume submission 

• Mock interview practice 

• Phone/campus screening 

• On-site interview 

• Negotiating an offer 


General Advice, on Page 17 

• Know the company & interviewers 

• Communicate clearly 

• Be passionate 

• Be honest 

• Stay positive 

• Don't apologize 

• Leave perks and money out 

• Be well-groomed 

• Mind your body language 

• Be ready for a stress interview 

• Learn from bad outcomes 

• Negotiate the best offer 


At the Interview, on Page 13 

• Don't solve the wrong problem 

• Get specs & requirements 

• Construct sample input/output 

• Work on concrete examples first 

• Spell out the brute-force solution 

• Think out loud 

• Apply patterns 

• Assume valid inputs 

• Test for corner-cases 

• Use proper syntax 

• Manage the whiteboard 

• Be aware of memory management 

• Get function signatures right 

Conducting an Interview, on Page 20 

• Don't be indecisive 

• Create a brand ambassador 

• Coordinate with other interviewers 
o know what to test on 

o look for patterns of mistakes 

• Characteristics of a good problem: 
o no single point of failure 

o has multiple solutions 
o covers multiple areas 
o is calibrated on colleagues 
o does not require unnecessary domain 
knowledge 

• Control the conversation 

o draw out quiet candidates 
o manage verbose/overconfident candi¬ 
dates 

• Use a process for recording & scoring 

• Determine what training is needed 

• Apply the litmus test 


The interview lifecycle 

Generally speaking, interviewing takes place in the following steps: 

(1.) Identify companies that you are interested in, and, ideally, find people you 
know at these companies. 

(2.) Prepare your resume using the guidelines on the facing page, and submit it via 
a personal contact (preferred), or through an online submission process or a 
campus career fair. 

(3.) Perform an initial phone screening, which often consists of a question-answer 
session over the phone or video chat with an engineer. You may be asked to 
submit code via a shared document or an online coding site such as ideone.com, 
collabedit.com, or coderpad.io. Don't take the screening casually—it can be ex¬ 
tremely challenging. 

(4.) Go for an on-site interview—this consists of a series of one-on-one interviews 
with engineers and managers, and a conversation with your Human Resources 
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Table 1.2: First read Chapter 4. For each chapter, first read its introductory text. Use textbooks for 
reference only. Unless a problem is italicized, it entails writing code. For Scenario i, write and test code 
for the problems in Columns 0 to i - 1, and pseudo-code for the problems in Column i. 


Scenario 1 

Scenario 2 

Scenario 3 

Scenario 4 

Hackathon 

Finals cram 

Term project 

Algorithms 

3 days 


7 days 

1 month 

4 months 

CO 

Cl 

C2 

C3 

C4 

5.1 

5.7 

5.8 

5.3,5.11 

5.9 

6.1, 6.6 

6.11, 6.17 

6.2,6.16 

6.5,6.8 

6.3, 6.9, 6.14 

7.1 

7.2, 7.4 

7.5, 7.6 

7.7, 7.8 

7.9, 7.11 

8.1 

8.2, 8.3 

8.4, 8.7 

8.10 

8.11 

9.1 

9.7 

9.2, 9.8 

9.3,9.9 

9.4 

10.1 

10.4 

10.2,10.12 

10.11 

10.13,10.16 

11.1 

11.4 

11.3 

11.5 

11.7 

12.1 

12.4,12.8 

12.3,12.9 

12.5,12.10 

12.6,12.7 

13.2 

13.6,13.3 

13.1,13.6 

13.4,13.7 

13.10 

14.1 

14.2 

14.4 

14.6,14.9 

14.7 

15.1 

15.2,15.3 

15.4, 15.8 

15.5,15.9 

15.7,15.10 

16.1 

16.2 

16.3 

16.4,16.9 

16.6,16.10 

17.1 

17.2 

17.3,17.6 

17.5,17.7 

17.12 

18.4 

18.6 

18.5,18.7 

18.8 

25.34 

19.1 

19.7 

19.2 

19.3 

19.9 

20.3 

20.6 

20.8 

20.9 

21.9 

21.13 

21.15 

21.16 

21.1 

21.2 


(HR) contact. 

(5.) Receive offers—these are usually a starting point for negotiations. 

Note that there may be variations—e.g., a company may contact you, or you 
may submit via your college's career placement center. The screening may involve 
a homework assignment to be done before or after the conversation. The on-site 
interview may be conducted over a video chat session. Most on-sites are half a day, 
but others may last the entire day. For anything involving interaction over a network, 
be absolutely sure to work out logistics (a quiet place to talk with a landline rather 
than a mobile, familiarity with the coding website and chat software, etc.) well in 
advance. 

We recommend that you interview at as many places as you can without it taking 
away from your job or classes. The experience will help you feel more comfortable 
with interviewing and you may discover you really like a company that you did not 
know much about. 

The resume 

It always astonishes us to see candidates who've worked hard for at least four years 
in school, and often many more in the workplace, spend 30 minutes jotting down 
random factoids about themselves and calling the result a resume. 

A resume needs to address HR staff, the individuals interviewing you, and the 
hiring manager. The HR staff, who typically first review your resume, look for 
keywords, so you need to be sure you have those covered. The people interviewing 
you and the hiring manager need to know what you've done that makes you special, 
so you need to differentiate yourself. 

Here are some key points to keep in mind when writing a resume: 
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(1.) Have a clear statement of your objective; in particular, make sure that you tailor 
your resume for a given employer. 

E.g., "My outstanding ability is developing solutions to computationally chal¬ 
lenging problems; communicating them in written and oral form; and working 
with teams to implement them. I would like to apply these abilities at XYZ." 

(2.) The most important points—the ones that differentiate you from everyone 
else—should come first. People reading your resume proceed in sequential 
order, so you want to impress them with what makes you special early on. 
(Maintaining a logical flow, though desirable, is secondary compared to this 
principle.) 

As a consequence, you should not list your programming languages, course- 
work, etc. early on, since these are likely common to everyone. You should 
list significant class projects (this also helps with keywords for HR.), as well 
as talks/papers you've presented, and even standardized test scores, if truly 
exceptional. 

(3.) The resume should be of a high-quality: no spelling mistakes; consistent spac- 
ings, capitalizations, numberings; and correct grammar and punctuation. Use 
few fonts. Portable Document Format (PDF) is preferred, since it renders well 
across platforms. 

(4.) Include contact information, a Linkedln profile, and, ideally, a URL to a personal 
homepage with examples of your work. These samples may be class projects, a 
thesis, and links to companies and products you've worked on. Include design 
documents as well as a link to your version control repository. 

(5.) If you can work at the company without requiring any special processing (e.g., 
if you have a Green Card, and are applying for a job in the US), make a note of 
that. 

(6.) Have friends review your resume; they are certain to find problems with it that 
you missed. It is better to get something written up quickly, and then refine it 
based on feedback. 

(7.) A resume does not have to be one page long—two pages are perfectly appro¬ 
priate. (Over two pages is probably not a good idea.) 

(8.) As a rule, we prefer not to see a list of hobbies/extracurricular activities (e.g., 
"reading books", "watching TV", "organizing tea party activities") unless they 
are really different (e.g., "Olympic rower") and not controversial. 

Whenever possible, have a friend or professional acquaintance at the company route 
your resume to the appropriate manager/HR contact—the odds of it reaching the 
right hands are much higher. At one company whose practices we are familiar with, 
a resume submitted through a contact is 50 times more likely to result in a hire than 
one submitted online. Don't worry about wasting your contact's time—employees 
often receive a referral bonus, and being responsible for bringing in stars is also 
viewed positively. 

Mock interviews 

Mock interviews are a great way of preparing for an interview. Get a friend to ask 
you questions (from EPI or any other source) and solve them on a whiteboard, with 
pen and paper, or on a shared document. Have your friend take notes and give you 
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feedback, both positive and negative. Make a video recording of the interview. You 
will cringe as you watch it, but it is better to learn of your mannerisms beforehand. 
Ask your friend to give hints when you get stuck. In addition to sharpening your 
problem solving and presentation skills, the experience will help reduce anxiety at 
the actual interview setting. If you cannot find a friend, you can still go through the 
same process, recording yourself. 

Language review 

Programs are written in Java 1.7. Some Java 1.7 specific constructs that we use are the 
diamond operator (<>), which reduces the verbosity of declaring and instantiating 
variables, the Ob j ects utility class, which eases writing hash functions and compara¬ 
tors, and binary literals and underscores, which make integral constants easier to 
read, e.g., ®b®®l_®®®®l. 

Usually we declare helper classes as static inner classes of the top-level class that 
provides the solution. You should be comfortable with the syntax used to instantiate 
an anonymous class, e.g., the comparator object in the sorting code on Page 235. 

Our test code lives within the main() method of the top-level class. We also have 
a Junit test suite, with one test per program which just wraps the each program's 
mainC). 

We have elected not to use popular third-party libraries, such as Apache Commons, 
and Google Guava—every program uses the JDK, and nothing else. We wrote a very 
small number of utility functions, largely for test code, e.g., fill and print methods. 
Apart from these, every solution file is self-contained. 

Wherever possible, you should use the standard library for basic containers. Some 
problems explicitly require you to write your own container classes. 

Best practices for interview code 

Now we describe practices we use in EPI that are not suitable for production code. 
They are necessitated by the finite time constraints of an interview. See Section 2 on 
Page 14 for more insights. 

• We make fields public, rather than use getters and setters. 

• We do not protect against invalid inputs, e.g., null references, negative entries 
in an array that's supposed to be all nonnegative, input streams that contain 
objects that are not of the expected type, etc. 

• We occasionally use static fields to pass values—this reduces the number of 
classes we need to write, at the cost of losing thread safety. 

• When asked to implement a container class, we do not provide generic solutions, 
e.g., we specialize the container to Integer, even when a type like ? extends 
Number will work. 

Now we describe practices we follow in EPI which are industry-standard, but we 
would not recommend for an interview. 

• We use long identifiers for pedagogy, e.g., queueOfMaximalUsers. In an actual 
interview, you should use shorter, less descriptive names than we have in our 
programs—writing queueOfMaximalUsers repeatedly is very time-consuming 
compared to writing q. 
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• We follow the Google Java style guide, which you should review before diving 
into EPI. The guide is fairly straightforward—it mostly addresses naming and 
spacing conventions, which should not be a high priority when presenting your 
solution. 

• We use the appropriate exception type, e.g.. No SuchElement Except ion when 
dequeing an empty queue. This is not needed in an interview, where throwing 
RuntimeException suffices. 

• When specifying types, we use the weakest type that provides the functionality 
we use, e.g., the argument to a function for computing the kth smallest element 
in an array is a Li st, rather than ArrayList. This is generally considered a best 
practice—it enables code reuse via polymorphism—but is not essential to use 
in an interview. (You should spend your time making sure the program works, 
rather than, e.g., if you should be using List or RandomAccess as the argument 
type.) 

An industry best practice that we use in EPI and recommend you use in an inter¬ 
view is explicitly creating classes for data clumps, i.e., groups of values that do not 
have any methods on them. Many programmers would use a generic Pair or Tuple 
class, but we have found that this leads to confusing and buggy programs. 

Books 

Our favorite Java reference is Peter Sestoft's "Java Precisely", which does a great job 
of covering the language constructs with examples. The definitive book for Java is 
"Java: The Complete Reference" by Oracle Press—this, along with the Java Language 
Specification, is the go-to resource for language lawyers. 

Joshua Bloch's "Effective Java" (second edition) is one of the best all-round pro¬ 
gramming books we have come across, addressing everything from the pitfalls of 
inheritance, to the Executor framework. For design patterns, we like "Head First 
Design Patterns" by Freeman et al.. Its primary drawback is its bulk. Tony Bevis' 
"Java Design Pattern Essentials" conveys the same content in a more succinct but 
less entertaining fashion. Note that programs for interviews are too short to take 
advantage of design patterns. 

"Java Concurrency in Practice", by Goetz et al. does a wonderful job of explaining 
pitfalls and practices around multithreaded Java programs. 
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Chapter 



Strategies For A Great Interview 


The essence of strategy is choosing what not to do. 


— M. E. Porter 


A typical one hour interview with a single interviewer consists of five minutes of 
introductions and questions about the candidate's resume. This is followed by five 
to fifteen minutes of questioning on basic programming concepts. The core of the 
interview is one or two detailed design questions where the candidate is expected 
to present a detailed solution on a whiteboard, paper, or integrated development 
environments (IDEs). Depending on the interviewer and the question, the solution 
may be required to include syntactically correct code and tests. 

Approaching the problem 

No matter how clever and well prepared you are, the solution to an interview problem 
may not occur to you immediately. Here are some things to keep in mind when this 
happens. 

Clarify the question: This may seem obvious but it is amazing how many in¬ 
terviews go badly because the candidate spends most of his time trying to solve the 
wrong problem. If a question seems exceptionally hard, you may have misunderstood 
it. 

A good way of clarifying the question is to state a concrete instance of the problem. 
For example, if the question is "find the first occurrence of a number greater than k in 
a sorted array", you could ask "if the input array is (2,20,30) and k is 3, then are you 
supposed to return 1, the index of 20?" These questions can be formalized as unit 
tests. 

Feel free to ask the interviewer what time and space complexity he would like in 
your solution. If you are told to implement an 0(ri) algorithm or use 0(1) space, it 
can simplify your life considerably. It is possible that he will refuse to specify these, 
or be vague about complexity requirements, but there is no harm in asking. Even if 
they are evasive, you may get some clues. 

Work on concrete examples: Consider the problem of determining the smallest 
amount of change that you cannot make with a given set of coins, as described on 
Page 30. This problem may seem difficult at first. However, if you try out the 
smallest amount that cannot be made with some small examples, e.g., {1,2}, {1,3}, 
{1,2,4}, {1,2,5}, you will get the key insights: examine coins in sorted order, and look 
for a large "jump"—a coin which is larger than the sum of the preceding coins. 
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Spell out the brute-force solution: Problems that are put to you in an interview 
tend to have an obvious brute-force solution that has a high time complexity com¬ 
pared to more sophisticated solutions. For example, instead of trying to work out 
a DP solution for a problem (e.g., for Problem 17.7 on Page 320), try all the possi¬ 
ble configurations. Advantages to this approach include: (1.) it helps you explore 
opportunities for optimization and hence reach a better solution, (2.) it gives you 
an opportunity to demonstrate some problem solving and coding skills, and (3.) it 
establishes that both you and the interviewer are thinking about the same problem. 
Be warned that this strategy can sometimes be detrimental if it takes a long time to 
describe the brute-force approach. 

Think out loud: One of the worst things you can do in an interview is to freeze 
up when solving the problem. It is always a good idea to think out loud and stay 
engaged. On the one hand, this increases your chances of finding the right solution 
because it forces you to put your thoughts in a coherent manner. On the other hand, 
this helps the interviewer guide your thought process in the right direction. Even if 
you are not able to reach the solution, the interviewer will form some impression of 
your intellectual ability. 

Apply patterns: Patterns—general reusable solutions to commonly occurring 
problems—can be a good way to approach a baffling problem. Examples include 
finding a good data structure, seeing if your problem is a good fit for a general al¬ 
gorithmic technique, e.g., divide-and-conquer, recursion, or dynamic programming, 
and mapping the problem to a graph. Patterns are described in Chapter 4. 

Presenting the solution 

Once you have an algorithm, it is important to present it in a clear manner. Your 
solution will be much simpler if you take advantage of libraries such as Java Collec¬ 
tions or C++ Boost. However, it is far more important that you use the language you 
are most comfortable with. Here are some things to keep in mind when presenting a 
solution. 

Libraries: Do not reinvent the wheel (unless asked to invent it). In particular, 
master the libraries, especially the data structures. For example, do not waste time 
and lose credibility trying to remember how to pass an explicit comparator to a BST 
constructor. Remember that a hash function should use exactly those fields which are 
used in the equality check. A comparison function should be transitive. 

Focus on the top-level algorithm: It's OK to use functions that you will implement 
later. This will let you focus on the main part of the algorithm, will penalize you less 
if you don't complete the algorithm. (Hash, equals, and compare functions are 
good candidates for deferred implementation.) Specify that you will handle main 
algorithm first, then corner cases. Add TODO comments for portions that you want 
to come back to. 

Manage the whiteboard: You will likely use more of the board than you expect, so 
start at the top-left comer. Make use of functions—skip implementing anything that's 
trivial (e.g., finding the maximum of an array) or standard (e.g., a thread pool). Best 
practices for coding on a whiteboard are very different from best practices for coding 
on a production project. For example, don't worry about skipping documentation, 
or using the right indentation. Writing on a whiteboard is much slower than on a 
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keyboard, so keeping your identifiers short (our recommendation is no more than 7 
characters) but recognizable is a best practice. Have a convention for identifiers, e.g., 
i , j , k for array indices, A, B, C for arrays, u, v, w for vectors, s for a String, sb for a 
StringBuilder, etc. 

Assume valid inputs: In a production environment, it is good practice to check 
if inputs are valid, e.g., that a string purporting to represent a nonnegative integer 
actually consists solely of numeric characters, no flight in a timetable arrives before 
it departs, etc. Unless they are part of the problem statement, in an interview setting, 
such checks are inappropriate: they take time to code, and distract from the core 
problem. (You should clarify this assumption with the interviewer.) 

Test for comer cases: For many problems, your general idea may work for most 
valid inputs but there may be pathological valid inputs where your algorithm (or 
your implementation of it) fails. For example, your binary search code may crash 
if the input is an empty array; or you may do arithmetic without considering the 
possibility of overflow. It is important to systematically consider these possibilities. 
If there is time, write unit tests. Small, extreme, or random inputs make for good 
stimuli. Don't forget to add code for checking the result. Occasionally, the code to 
handle obscure corner cases may be too complicated to implement in an interview 
setting. If so, you should mention to the interviewer that you are aware of these 
problems, and could address them if required. 

Syntax: Interviewers rarely penalize you for small syntax errors since modern 
IDE excel at handling these details. However, lots of bad syntax may result in the 
impression that you have limited coding experience. Once you are done writing your 
program, make a pass through it to fix any obvious syntax errors before claiming you 
are done. 

Candidates often tend to get function signatures wrong and it reflects poorly on 
them. For example, it would be an error to write a function in C that returns an array 
but not its size. 

Memory management: Generally speaking, it is best to avoid memory manage¬ 
ment operations altogether. See if you can reuse space. For example, some linked 
list problems can be solved with (9(1) additional space by reusing existing nodes. 

Your Interviewer Is Not Alan Turing: Interviewers are not capable of analyzing 
long programs, particularly on a whiteboard or paper. Therefore, they ask questions 
whose solutions use short programs. A good tip is that if your solution takes more 
than 50-70 lines to code, it's a sign that you are on the wrong track, and you should 
reconsider your approach. 

Know your interviewers & the company 

It can help you a great deal if the company can share with you the background of 
your interviewers in advance. You should use search and social networks to learn 
more about the people interviewing you. Letting your interviewers know that you 
have researched them helps break the ice and forms the impression that you are 
enthusiastic and will go the extra mile. For fresh graduates, it is also important to 
think from the perspective of the interviewers as described in Chapter 3. 

Once you ace your interviews and have an offer, you have an important decision 
to make—is this the organization where you want to work? Interviews are a great 
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time to collect this information. Interviews usually end with the interviewers letting 
the candidates ask questions. You should make the best use of this time by getting 
the information you would need and communicating to the interviewer that you are 
genuinely interested in the job. Based on your interaction with the interviewers, you 
may get a good idea of their intellect, passion, and fairness. This extends to the team 
and company. 

In addition to knowing your interviewers, you should know about the company 
vision, history, organization, products, and technology. You should be ready to talk 
about what specifically appeals to you, and to ask intelligent questions about the 
company and the job. Prepare a list of questions in advance; it gets you helpful 
information as well as shows your knowledge and enthusiasm for the organization. 
You may also want to think of some concrete ideas around things you could do for 
the company; be careful not to come across as a pushy know-it-all. 

All companies want bright and motivated engineers. However, companies differ 
greatly in their culture and organization. Here is a brief classification. 

Mature consumer-facing company, e.g., Google: wants candidates who under¬ 
stand emerging technologies from the user's perspective. Such companies have a 
deeper technology stack, much of which is developed in-house. They have the re¬ 
sources and the time to train a new hire. 

Enterprise-oriented company, e.g., Oracle: looks for developers familiar with 
how large projects are organized, e.g., engineers who are familiar with reviews, 
documentation, and rigorous testing. 

Government contractor, e.g., Lockheed-Martin: values knowledge of specifi¬ 
cations and testing, and looks for engineers who are familiar with government- 
mandated processes. 

Startup, e.g., Uber: values engineers who take initiative and develop products on 
their own. Such companies do not have time to train new hires, and tend to hire can¬ 
didates who are very fast learners or are already familiar with their technology stack, 
e.g., their web application framework, machine learning system, etc. Embedded sys¬ 
tems/chip design company, e.g.. National Instruments: wants software engineers 
who know enough about hardware to interface with the hardware engineers. The 
tool chain and development practices at such companies tend to be very mature. 

General conversation 

Often interviewers will ask you questions about your past projects, such as a senior 
design project or an internship. The point of this conversation is to answer the 
following questions: 

Can the candidate clearly communicate a complex idea? This is one of the most 
important skills for working in an engineering team. If you have a grand idea to 
redesign a big system, can you communicate it to your colleagues and bring them 
on board? It is crucial to practice how you will present your best work. Being 
precise, clear, and having concrete examples can go a long way here. Candidates 
communicating in a language that is not their first language, should take extra care 
to speak slowly and make more use of the whiteboard to augment their words. 

Is the candidate passionate about his work? We always want our colleagues to 
be excited, energetic, and inspiring to work with. If you feel passionately about your 
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work, and your eyes light up when describing what you've done, it goes a long way 
in establishing you as a great colleague. Hence, when you are asked to describe a 
project from the past, it is best to pick something that you are passionate about rather 
than a project that was complex but did not interest you. 

Is there a potential interest match with some project? The interviewer may gauge 
areas of strengths for a potential project match. If you know the requirements of the 
job, you may want to steer the conversation in that direction. Keep in mind that 
because technology changes so fast many teams prefer a strong generalist, so don't 
pigeonhole yourself. 

Other advice 

A bad mental and physical attitude can lead to a negative outcome. Don't let these 
simple mistakes lead to your years of preparation going to waste. 

Be honest: Nobody wants a colleague who falsely claims to have tested code or 
done a code review. Dishonesty in an interview is a fast pass to an early exit. 

Remember, nothing breaks the truth more than stretching it—you should be ready 
to defend anything you claim on your resume. If your knowledge of Python extends 
only as far as having cut-and-paste sample code, do not add Python to your resume. 

Similarly, if you have seen a problem before, you should say so. (Be sure that it 
really is the same problem, and bear in mind you should describe a correct solution 
quickly if you claim to have solved it before.) Interviewers have been known to 
collude to ask the same question of a candidate to see if he tells the second interviewer 
about the first instance. An interviewer may feign ignorance on a topic he knows in 
depth to see if a candidate pretends to know it. 

Keep a positive spirit: A cheerful and optimistic attitude can go a long way. 
Absolutely nothing is to be gained, and much can be lost, by complaining how 
difficult your journey was, how you are not a morning person, how inconsiderate the 
airline/hotel/HR staff were, etc. 

Don't apologize: Candidates sometimes apologize in advance for a weak GPA, 
rusty coding skills, or not knowing the technology stack. Their logic is that by being 
proactive they will somehow benefit from lowered expectations. Nothing can be 
further from the truth. It focuses attention on shortcomings. More generally, if you 
do not believe in yourself, you cannot expect others to believe in you. 

Keep money and perks out of the interview: Money is a big element in any job 
but it is best left discussed with the HR division after an offer is made. The same is 
true for vacation time, day care support, and funding for conference travel. 

Appearance: Most software companies have a relaxed dress-code, and new grad¬ 
uates may wonder if they will look foolish by overdressing. The damage done when 
you are too casual is greater than the minor embarrassment you may feel at being 
overdressed. It is always a good idea to err on the side of caution and dress formally 
for your interviews. At the minimum, be clean and well-groomed. 

Be aware of your body language: Think of a friend or coworker slouched all the 
time or absentmindedly doing things that may offend others. Work on your posture, 
eye contact and handshake, and remember to smile. 
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Stress interviews 

Some companies, primarily in the finance industry, make a practice of having one 
of the interviewers create a stressful situation for the candidate. The stress may be 
injected technically, e.g., via a ninja problem, or through behavioral means, e.g., the 
interviewer rejecting a correct answer or ridiculing the candidate. The goal is to see 
how a candidate reacts to such situations—does he fall apart, become belligerent, or 
get swayed easily. The guidelines in the previous section should help you through a 
stress interview. (Bear in mind you will not know a priori if a particular interviewer 
will be conducting a stress interview.) 

Learning from bad outcomes 

The reality is that not every interview results in a job offer. There are many reasons 
for not getting a particular job. Some are technical: you may have missed that key 
flash of insight, e.g., the key to solving the maximum-profit on Page 1 in linear time. 
If this is the case, go back and solve that problem, as well as related problems. 

Often, your interviewer may have spent a few minutes looking at your resume— 
this is a depressingly common practice. This can lead to your being asked questions 
on topics outside of the area of expertise you claimed on your resume, e.g., routing 
protocols or Structured Query Language (SQL). If so, make sure your resume is 
accurate, and brush up on that topic for the future. 

You can fail an interview for nontechnical reasons, e.g., you came across as un¬ 
interested, or you did not communicate clearly. The company may have decided 
not to hire in your area, or another candidate with similar ability but more relevant 
experience was hired. 

You will not get any feedback from a bad outcome, so it is your responsibility to 
try and piece together the causes. Remember the only mistakes are the ones you don't 
learn from. 

Negotiating an offer 

An offer is not an offer till it is on paper, with all the details filled in. All offers are 
negotiable. We have seen compensation packages bargained up to twice the initial 
offer, but 10-20% is more typical. When negotiating, remember there is nothing to be 
gained, and much to lose, by being rude. (Being firm is not the same as being rude.) 

To get the best possible offer, get multiple offers, and be flexible about the form of 
your compensation. For example, base salary is less flexible than stock options, sign- 
on bonus, relocation expenses, and Immigration and Naturalization Service (INS) 
filing costs. Be concrete—instead of just asking for more money, ask for a P% higher 
salary. Otherwise the recruiter will simply come back with a small increase in the 
sign-on bonus and claim to have met your request. 

Your HR contact is a professional negotiator, whose fiduciary duty is to the com¬ 
pany. He will know and use negotiating techniques such as reciprocity, getting 
consensus, putting words in your mouth ("don't you think that's reasonable?"), as 
well as threats, to get the best possible deal for the company. (This is what recruiters 
themselves are evaluated on internally.) The Wikipedia article on negotiation lays 
bare many tricks we have seen recruiters employ. 
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One suggestion: stick to email, where it is harder for someone to paint you into 
a comer. If you are asked for something (such as a copy of a competing offer), get 
something in return. Often it is better to bypass the HR contact and speak directly 
with the hiring manager. 

At the end of the day, remember your long term career is what counts, and joining 
a company that has a brighter future (social-mobile vs. legacy enterprise), or offers 
a position that has more opportunities to rise (developer vs. tester) is much more 
important than a 10-20% difference in compensation. 
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Chapter 



Conducting An Interview 

Translated—"If you know both yourself and 
your enemy, you can win numerous battles with¬ 
out jeopardy." 

— "The Art of War," 
Sun Tzu, 515 B.C. 


In this chapter we review practices that help interviewers identify a top hire. We 
strongly recommend interviewees read it—knowing what an interviewer is looking 
for will help you present yourself better and increase the likelihood of a successful 
outcome. 

For someone at the beginning of their career, interviewing may feel like a huge 
responsibility. Hiring a bad candidate is expensive for the organization, not just 
because the hire is unproductive, but also because he is a drain on the productivity 
of his mentors and managers, and sets a bad example. Firing someone is extremely 
painful as well as bad for to the morale of the team. On the other hand, discarding 
good candidates is problematic for a rapidly growing organization. Interviewers 
also have a moral responsibility not to unfairly crush the interviewee's dreams and 
aspirations. 

Objective 

The ultimate goal of any interview is to determine the odds that a candidate will 
be a successful employee of the company. The ideal candidate is smart, dedicated, 
articulate, collegial, and gets things done quickly, both as an individual and in a team. 
Ideally, your interviews should be designed such that a good candidate scores 1.0 and 
a bad candidate scores 0.0. 

One mistake, frequently made by novice interviewers, is to be indecisive. Unless 
the candidate walks on water or completely disappoints, the interviewer tries not to 
make a decision and scores the candidate somewhere in the middle. This means that 
the interview was a wasted effort. 

A secondary objective of the interview process is to turn the candidate into a brand 
ambassador for the recruiting organization. Even if a candidate is not a good fit for 
the organization, he may know others who would be. It is important for the candidate 
to have an overall positive experience during the process. It seems obvious that it is 
a bad idea for an interviewer to check email while the candidate is talking or insult 
the candidate over a mistake he made, but such behavior is depressingly common. 
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Outside of a stress interview, the interviewer should work on making the candidate 
feel positively about the experience, and, by extension, the position and the company. 

What to ask 

One important question you should ask yourself as an interviewer is how much 
training time your work environment allows. For a startup it is important that a 
new hire is productive from the first week, whereas a larger organization can budget 
for several months of training. Consequently, in a startup it is important to test the 
candidate on the specific technologies that he will use, in addition to his general 
abilities. 

For a larger organization, it is reasonable not to emphasize domain knowledge 
and instead test candidates on data structures, algorithms, system design skills, and 
problem solving techniques. The justification for this is as follows. Algorithms, data 
structures, and system design underlie all software. Algorithms and data structure 
code is usually a small component of a system dominated by the user interface (UI), 
input/output (I/O), and format conversion. It is often hidden in library calls. However, 
such code is usually the crucial component in terms of performance and correctness, 
and often serves to differentiate products. Furthermore, platforms and programming 
languages change quickly but a firm grasp of data structures, algorithms, and system 
design principles, will always be a foundational part of any successful software 
endeavor. Finally, many of the most successful software companies have hired based 
on ability and potential rather than experience or knowledge of specifics, underlying 
the effectiveness of this approach to selecting candidates. 

Most big organizations have a structured interview process where designated 
interviewers are responsible for probing specific areas. For example, you may be 
asked to evaluate the candidate on their coding skills, algorithm knowledge, critical 
thinking, or the ability to design complex systems. This book gives interviewers 
access to a fairly large collection of problems to choose from. When selecting a 
problem keep the following in mind: 

No single point of failure —if you are going to ask just one question, you should 
not pick a problem where the candidate passes the interview if and only if he gets 
one particular insight. The best candidate may miss a simple insight, and a mediocre 
candidate may stumble across the right idea. There should be at least two or three 
opportunities for the candidates to redeem themselves. For example, problems that 
can be solved by dynamic programming can almost always be solved through a 
greedy algorithm that is fast but suboptimum or a brute-force algorithm that is slow 
but optimum. In such cases, even if the candidate cannot get the key insight, he can 
still demonstrate some problem solving abilities. Problem 6.6 on Page 70 exemplifies 
this type of question. 

Multiple possible solutions —if a given problem has multiple solutions, the 
chances of a good candidate coming up with a solution increases. It also gives the 
interviewer more freedom to steer the candidate. A great candidate may finish with 
one solution quickly enough to discuss other approaches and the trade-offs between 
them. For example. Problem 12.9 on Page 202 can be solved using a hash table or a 
bit array; the best solution makes use of binary search. 


21 



Cover multiple areas —even if you are responsible for testing the candidate on 
algorithms, you could easily pick a problem that also exposes some aspects of design 
and software development. For example. Problem 20.8 on Page 385 tests candidates 
on concurrency as well as data structures. Problem 6.15 on Page 83 requires knowl¬ 
edge of both probability and binary search. 

Calibrate on colleagues —interviewers often have an incorrect notion of how dif¬ 
ficult a problem is for a thirty minute or one hour interview. It is a good idea to check 
the appropriateness of a problem by asking one of your colleagues to solve it and 
seeing how much difficulty they have with it. 

No unnecessary domain knowledge —it is not a good idea to quiz a candidate on 
advanced graph algorithms if the job does not require it and the candidate does not 
claim any special knowledge of the field. (The exception to this rule is if you want to 
test the candidate's response to stress.) 

Conducting the interview 

Conducting a good interview is akin to juggling. At a high level, you want to ask 
your questions and evaluate the candidate's responses. Many things can happen in 
an interview that could help you reach a decision, so it is important to take notes. At 
the same time, it is important to keep a conversation going with the candidate and 
help him out if he gets stuck. Ideally, have a series of hints worked out beforehand, 
which can then be provided progressively as needed. Coming up with the right set 
of hints may require some thinking. You do not want to give away the problem, yet 
find a way for the candidate to make progress. Here are situations that may throw 
you off: 

A candidate that gets stuck and shuts up: Some candidates get intimidated by 
the problem, the process, or the interviewer, and just shut up. In such situations, a 
candidate's performance does not reflect his true caliber. It is important to put the 
candidate at ease, e.g., by beginning with a straightforward question, mentioning 
that a problem is tough, or asking them to think out loud. 

A verbose candidate: Candidates who go off on tangents and keep on talking 
without making progress render an interview ineffective. Again, it is important to 
take control of the conversation. For example you could assert that a particular path 
will not make progress. 

An overconfident candidate: It is common to meet candidates who weaken their 
case by defending an incorrect answer. To give the candidate a fair chance, it is 
important to demonstrate to him that he is making a mistake, and allow him to 
correct it. Often the best way of doing this is to construct a test case where the 
candidate's solution breaks down. 

Scoring and reporting 

At the end of an interview, the interviewers usually have a good idea of how the 
candidate scored. However, it is important to keep notes and revisit them before 
making a final decision. Whiteboard snapshots and samples of any code that the 
candidate wrote should also be recorded. You should standardize scoring based on 
which hints were given, how many questions the candidate was able to get to, etc. 
Although isolated minor mistakes can be ignored, sometimes when you look at all 
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the mistakes together, clear signs of weakness in certain areas may emerge, such as a 
lack of attention to detail and unfamiliarity with a language. 

When the right choice is not clear, wait for the next candidate instead of possibly 
making a bad hiring decision. The litmus test is to see if you would react positively 
to the candidate replacing a valuable member of your team. 
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Chapter 


■ Problem Solving 

It's not that I'm so smart, it's just that I stay with problems longer. 

— A. Einstein 

In this chapter we describe approaches to solving programming problems that can 
help you when you are faced with a tricky interview problem. Specifically, we 
cover key data structures in Section 4.1, we present algorithm patterns in Section 4.2 
on Page 30, we describe ideas and notation from complexity theory in Section 4.3 
on Page 40, and we discuss strategies for coping with computationally intractable 
problems in Section 4.4 on Page 41. 

Bear in mind developing problem solving skills is like learning to play a musical 
instrument—books and teachers can point you in the right direction, but only your 
hard work will take you there. Just as a musician, you need to know underlying 
concepts, but theory is no substitute for practice. It is precisely for this reason that 
EPI focuses on problems. 

4.1 Data structure review 

A data structure is a particular way of storing and organizing related data items so that 
they can be manipulated efficiently. Usually, the correct selection of data structures 
is key to designing a good algorithm. Different data structures are suited to different 
applications; some are highly specialized. For example, heaps are particularly well- 
suited for algorithms that merge sorted data streams, while compiler implementations 
usually use hash tables to lookup identifiers. 

The data structures described in this chapter are the ones commonly used. Other 
data structures, such as skip lists, treaps, Fibonacci heaps, tries, and disjoint-set data 
structures, have more specialized applications. 

Solutions often require a combination of data structures. For example, tracking the 
most visited pages on a website involves a combination of a heap, a queue, a binary 
search tree, and a hash table. See Solution 25.21 on Page 469 for details. 

Primitive types 

You should be comfortable with the basic types (chars, integers, doubles, etc.), their 
variants (unsigned, long, etc.), and operations on them (bitwise operators, compar¬ 
ison, etc.). Don't forget that the basic types differ among programming languages. 
For example, Java has no unsigned integers, and the integer width is compiler- and 
machine-dependent in C. 
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Table 4.1: Data structures. 


Data structure 

Primitive types 

Arrays 

Strings 

Lists 


Stacks and queues 


Binary trees 


Heaps 


Hash tables 


Binary search trees 


Key points 

Know how int, char, double, etc. are represented in 
memory and the primitive operations on them. 

Fast access for element at an index, slow lookups (un¬ 
less sorted) and insertions. Be comfortable with no¬ 
tions of iteration, resizing, partitioning, merging, etc. 

Know how strings are represented in memory. Under¬ 
stand basic operators such as comparison, copying, 
matching, joining, splitting, etc. 

Understand trade-offs with respect to arrays. Be com¬ 
fortable with iteration, insertion, and deletion within 
singly and doubly linked lists. Know how to imple¬ 
ment a list with dynamic allocation, and with arrays. 

Recognize where last-in first-out (stack) and first-in 
first-out (queue) semantics are applicable. Know array 
and linked list implementations. 

Use for representing hierarchical data. Know about 
depth, height, leaves, search path, traversal sequences, 
successor/predecessor operations. 

Key benefit: 0(1) lookup find-max, 0(log n) insertion, 
and <9(log n) deletion of max. Node and array repre¬ 
sentations. Min-heap variant. 

Key benefit: 0(1) insertions, deletions and lookups. 
Key disadvantages: not suitable for order-related 
queries; need for resizing; poor worst-case perfor¬ 
mance. Understand implementation using array of 
buckets and collision chains. Know hash functions for 
integers, strings, objects. 

Key benefit: O(logn) insertions, deletions, lookups, 
find-min, find-max, successor, predecessor when tree 
is height-balanced. Understand node fields, pointer 
implementation. Be familiar with notion of balance, 
and operations maintaining balance. 


A common problem related to basic types is computing the number of bits set to 1 
in an integer-valued variable x. To solve this problem you need to know how to ma¬ 
nipulate individual bits in an integer. One straightforward approach is to iteratively 
test individual bits using the value 1 as a bitmask. Specifically, we iteratively identify 
bits of x that are set to 1 by examining the bitwise-AND of x with the bitmask, shifting 
x right one bit at a time. The overall complexity is 0(n) where n is the length of the 
integer. 

Another approach, which may run faster on some inputs, is based on computing 
y = x & ~(x-1), where & is the bitwise-AND operator and ~ is the bitwise complement 
operator. The variable y is 1 at exactly the lowest bit of x that is 1; all other bits in y are 
0. For example, if x = (00101100)2, then x - 1 = (00101011)2, ~(* - 1) = (11010100)2, 
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and y = (00101100) 2 & (11010100) 2 = (00000100) 2 . This calculation is robust—it is 
correct for unsigned and two's-complement representations. Consequently, this bit 
may be removed from x by computing x © y, where © is the bitwise-XOR function. 
The time complexity is 0(s), where s is the number of bits set to 1 in x. 

The fact that x & ~(x - 1) isolates the lowest bit that is 1 in x is important enough 
that you should memorize it. However, it is also fairly easy to derive. First, suppose 
x is not 0, i.e., it has has a bit that is one. Subtracting one from x changes the rightmost 
bit to zero and sets all the lower bits to one (if you add one now, you get the original 
value back). The effect is to mask out the rightmost one. Therefore x & ~(x - 1) has a 
single bit set to one, namely, the rightmost 1 in x. Now suppose x is 0. Subtracting 
one from x underflows, resulting in a word in which all bits are set to one. Again, 
x & ~(x - 1) is 0. 

A similar derivation shows that x &(x - 1) replaces the lowest bit that is 1 with 
0. For example, if x = (00101100) 2 , then x - 1 = (00101011) 2 , so x &(x - 1) = 
(00101100) 2 &(00101011) 2 = (00101000) 2 . This fact can also be very useful. 

Consider sharpening your bit manipulation skills by writing expressions that use 
bitwise operators, equality checks, and Boolean operators to do the following. 

• Right propagate the rightmost set bit in x, e.g., turns (01010000) 2 to (01011111) 2 . 

• Compute x modulo a power of two, e.g., returns 13 for 77 mod 64. 

• Test if x is a power of 2, i.e., evaluates to true for x = 1,2,4,8,..., false for all 
other values. 

In practice, if the computation is done repeatedly, the most efficient approach 
would be to create a lookup table. In this case, we could use a 65536 entry 
integer-valued array P, such that P[i] is the number of bits set to 1 in i. If x is 
64 bits, the result can be computed by decomposing x into 4 disjoint 16-bit words, 
h3, hi, hi, and hO. The 16-bit words are computed using bitmasks and shifting, e.g., 
hi is (x » 16 & (1111111111111111) 2 ). The final result is P[h3] + P[h2] + P[hl] + P[h0]. 
Computing the parity of an integer is closely related to counting the number of bits 
set to 1, and we present a detailed analysis of the parity problem in Solution 5.1 on 
Page 46. 

Problems involving manipulation of bit-level data are often asked in interviews. It 
is easy to introduce errors in code that manipulates bit-level data; as the saying goes, 
when you play with bits, expect to get bitten. 

Arrays 

Conceptually, an array maps integers in the range [0, n — 1] to objects of a given type, 
where n is the number of objects in this array. Array lookup and insertion are fast, 
making arrays suitable for a variety of applications. Reading past the last element of 
an array is a common error, invariably with catastrophic consequences. 

The following problem arises when optimizing quicksort: given an array A whose 
elements are comparable, and an index i, reorder the elements of A so that the initial 
elements are all less than A[i], and are followed by elements equal to A[i], which in 
turn are followed by elements greater than A[i], using 0(1) space. 

The key to the solution is to maintain two regions on opposite sides of the array 
that meet the requirements, and expand these regions one element at a time. Details 
are given in Solution 6.1 on Page 63. 
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Strings 


A string can be viewed as a special kind of array, namely one made out of charac¬ 
ters. We treat strings separately from arrays because certain operations which are 
commonly applied to strings—for example, comparison, joining, splitting, searching 
for substrings, replacing one string by another, parsing, etc.—do not make sense for 
general arrays. 

Our solution to the look-and-say problem illustrates operations on strings. The 
look-and-say sequence begins with 1; the subsequent integers describe the digits 
appearing in the previous number in the sequence. The first eight integers in the 
look-and-say sequence are <1,11,21,1211,111221,312211,13112221,1113213211). The 
look-and-say problem entails computing the nth integer in this sequence. Although 
the problem is cast in terms of integers, the string representation is far more convenient 
for counting digits. Details are given in Solution 7.8 on Page 104. 

Lists 

A list implements an ordered collection of values, which may include repetitions. In 
the context of this book we view a list as a sequence of nodes where each node has a 
link to the next node in the sequence. In a doubly linked list each node also has a link 
to the prior node. 

A list is similar to an array in that it contains objects in a linear order. The key 
differences are that inserting and deleting elements in a list has time complexity 
0(1). On the other hand, obtaining the kth element in a list is expensive, having 0(n) 
time complexity. Lists are usually building blocks of more complex data structures. 
However, they can be the subject of tricky problems in their own right, as illustrated 
by the following: 

Let L be a singly linked list. Assume its nodes are numbered starting at 0. Define 
the zip of L to be the list consisting of the interleaving of the nodes numbered 0,1,2,... 
with the nodes numbered n - 1, n - 2, n - 3,..., where n is the number of nodes in the 
list. 

Suppose you were asked to write a program that computes the zip of a list, with 
the constraint that it uses 0(1) space. The operation of this program is illustrated in 
Figure 4.1. 
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Figure 4.1: Zipping a list. 
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The solution is based on an appropriate iteration combined with "pointer swap¬ 
ping", i.e., updating next field for each node. Refer to Solution 25.9 on Page 448 for 
details. 

Stacks and queues 

Stacks support last-in, first-out semantics for inserts and deletes, whereas queues 
are first-in, first-out. Both are commonly implemented using linked lists or arrays. 
Similar to lists, stacks and queues are usually building blocks in a solution to a 
complex problem, but can make for interesting problems in their own right. 

As an example consider the problem of evaluating Reverse Polish notation expres¬ 
sions, i.e., expressions of the form "3,4,X, 1,2, +,+", "1,1,+,—2, X", or "4,6,/, 2,/". 
A stack is ideal for this purpose—operands are pushed on the stack, and popped as 
operators are processed, with intermediate results being pushed back onto the stack. 
Details are given in Solution 9.2 on Page 136. 

Binary trees 

A binary tree is a data structure that is used to represent hierarchical relationships. 
Binary trees are the subject of Chapter 10. Binary trees most commonly occur in the 
context of binary search trees, wherein keys are stored in a sorted fashion. However, 
there are many other applications of binary trees. Consider a set of resources orga¬ 
nized as nodes in a binary tree. Processes need to lock resource nodes. A node may 
be locked if and only if none of its descendants and ancestors are locked. Your task is 
to design and implement an application programming interface (API) for locking. 

A reasonable API is one with a method for checking if a node is locked, and meth¬ 
ods for locking and unlocking a node. Naively implemented, the time complexity for 
these methods is 0(n), where n is the number of nodes. However, these can be made 
to run in time 0(1), 0(h), and 0(h), respectively, where h is the height of the tree, if 
nodes have a parent field. Details are given in Solution 10.17 on Page 173. 

Heaps 

A heap (also known as a priority queue) is a data structure based on a binary tree. A 
heap resembles a queue, with one difference: each element has a "priority" associated 
with it, and deletion removes the element with the highest priority. 

Let's say you are given a set of files, each containing stock trade information. Each 
trade appears as a separate line containing information about that trade. Lines begin 
with an integer-valued timestamp, and lines within a file are sorted in increasing 
order of timestamp. Suppose you were asked to design an algorithm that combines 
the set of files into a single file R in which trades are sorted by timestamp. 

This problem can be solved by a multistage merge process, but there is a trivial 
solution based on a min-heap data structure. Entries are trade-file pairs and are 
ordered by the timestamp of the trade. Initially, the min-heap contains the first trade 
from each file. Iteratively delete the minimum entry e = (t,f) from the min-heap, 
write t to R, and add in the next entry in the file /. Details are given in Solution 11.1 
on Page 177. 
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Hash tables 


A hash table is a data structure used to store keys, optionally, with corresponding 
values. Inserts, deletes and lookups run in 0(1) time on average. One caveat is that 
these operations require a good hash function—a mapping from the set of all possible 
keys to the integers which is similar to a uniform random assignment. Another caveat 
is that if the number of keys that is to be stored is not known in advance then the 
hash table needs to be periodically resized, which, depending on how the resizing is 
implemented, can lead to some updates having 0(n) complexity. 

Suppose you were asked to write a program which takes a string s as input, and 
returns true if the characters in s can be permuted to form a string that is palindromic, 
i.e., reads the same backwards as forwards. For example, your program should return 
true for "GATTAACAG", since "GATACATAG" is a permutation of this string and is 
palindromic. Working through examples, you should see that a string is palindromic 
if and only if each character appears an even number of times, with possibly a single 
exception, since this allows for pairing characters in the first and second halves. 

A hash table makes performing this test trivial. We build a hash table H whose 
keys are characters, and corresponding values are the number of occurrences for 
that character. The hash table H is created with a single pass over the string. After 
computing the number of occurrences, we iterate over the key-value pairs in H. If 
more than one character has an odd count, we return false; otherwise, we return true. 
Details are given in Solution 13.1 on Page 212. 

Suppose you were asked to write an application that compares n programs for 
plagiarism. Specifically, your application is to break every program into overlapping 
character strings, each of length 100, and report on the number of strings that appear 
in each pair of programs. A hash table can be used to perform this check very 
efficiently if the right hash function is used. Details are given in Solution 21.3 on 
Page 392. 

Binary search trees 

Binary search trees (BSTs) are used to store objects that are comparable. BSTs are the 
subject of Chapter 15. The underlying idea is to organize the objects in a binary tree 
in which the nodes satisfy the BST property on Page 254. Insertion and deletion can 
be implemented so that the height of the BST is <9(log n), leading to fast (<9(log rij) 
lookup and update times. AVL trees and red-black trees are BST implementations 
that support this form of insertion and deletion. 

BSTs are a workhorse of data structures and can be used to solve almost every 
data structures problem reasonably efficiently. It is common to augment the BST to 
make it possible to manipulate more complicated data, e.g., intervals, and efficiently 
support more complex queries, e.g., the number of elements in a range. 

As an example application of BSTs, consider the following problem. You are given 
a set of line segments. Each segment is a closed interval [/,, r,] of the X-axis, a color, 
and a height. For simplicity assume no two segments whose intervals overlap have 
the same height. When the X-axis is viewed from above the color at point x on the 
X-axis is the color of the highest segment that includes x. (If no segment contains x. 
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the color is blank.) You are to implement a function that computes the sequence of 
colors as seen from the top. 

The key idea is to sort the endpoints of the line segments and do a scan from 
left-to-right. As we do the scan, we maintain a list of line segments that intersect the 
current position as well as the highest line and its color. To quickly lookup the highest 
line in a set of intersecting lines we keep the current set in a BST, with the interval's 
height as its key. Details are given in Solution 25.25 on Page 476. 

4.2 Algorithm patterns 

We describe algorithm design patterns that we have found to be helpful in solving 
interview problems. These patterns fall into two categories—analysis patterns, sum¬ 
marized in Table 4.2, which yield insight into the problem, and algorithm design 
patterns, summarized in Table 4.3 on the facing page, which provide skeletal code. 
Keep in mind that you may have to use a combination of approaches to solve a 
problem. 


Analysis principle 

Concrete examples 

Case analysis 
Iterative refinement 
Reduction 
Graph modeling 


Concrete examples 

Problems that seem difficult to solve in the abstract can become much more tractable 
when you examine concrete instances. Specifically, the following types of inputs can 
offer tremendous insight: 

• small inputs, such as an array or a BST containing 5-7 elements. 

• extreme/specialized inputs, e.g., binary values, nonoverlapping intervals, 
sorted arrays, connected graphs, etc. 

Problems 6.10 on Page 76 and 16.1 on Page 283 are illustrative of small inputs, and 
Problems 25.1 on Page 435, 7.4 on Page 98,12.8 on Page 199, and 25.28 on Page 483 
are illustrative of extreme/specialized inputs. 

Here is an example that shows the power of small concrete example analysis. 
Given a set of coins, there are some amounts of change that you may not be able to 


Table 4.2: Analysis patterns. 

Key points 

Manually solve concrete instances of the problem and 
then build a general solution. 

Split the input/execution into a number of cases and 
solve each case in isolation. 

Most problems can be solved using a brute-force ap¬ 
proach. Find such a solution and improve upon it. 

Use a well-known solution to some other problem as a 
subroutine. 

Describe the problem using a graph and solve it using 
an existing algorithm. 
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Table 4.3: Algorithm design patterns. 


Technique 


Key points 


Sorting 

Recursion 

Divide-and-conquer 
Dynamic programming 
Greedy algorithms 


Uncover some structure by sorting the input. 

If the structure of the input is defined in a recursive 
manner, design a recursive algorithm that follows the 
input definition. 

Divide the problem into two or more smaller inde¬ 
pendent subproblems and solve the original problem 
using solutions to the subproblems. 

Compute solutions for smaller instances of a given 
problem and use these solutions to construct a solu¬ 
tion to the problem. Cache for performance. 

Compute a solution in stages, making choices that are 
locally optimum at step; these choices are never un¬ 
done. 


Invariants Identify an invariant and use it to rule out potential 

solutions that are suboptimal/dominated by other so¬ 
lutions. 


make with them, e.g., you cannot create a change amount greater than the sum of the 
your coins. For example, if your coins are 1,1,1,1,1,5,10,25, then the smallest value 
of change which cannot be made is 21. 

Suppose you were asked to write a program which takes an array of positive 
integers and returns the smallest number which is not to the sum of a subset of 
elements of the array. 

A brute-force approach would be to enumerate all possible values, starting from 
1, testing each value to see if it is the sum of array elements. However, there is no 
simple efficient algorithm for checking if a given number is the sum of a subset of 
entries in the array. Heuristics may be employed, but the program will be unwieldy. 

We can get a great deal of insight from some small concrete examples. Observe 
that (1,2) produces 1,2,3, and (1,3) produces 1,3,4. A trivial observation is that the 
smallest element in the array sets a lower bound on the change amount that can be 
constructed from that array, so if the array does not contain a 1, it cannot produce 1. 
However, it may be possible to produce 2 without a 2 being present, since there can 
be 2 or more Is present. 

Continuing with a larger example, (1,2,4) produces 1,2,3,4,5,6,7, and (1,2,5) 
produces 1,2,3,5,6,7,8. Generalizing, suppose a collection of numbers can produce 
every value up to and including V, but not V + 1. Now consider the effect of adding 
a new element u to the collection. If u < V + 1, we can still produce every value up to 
and including V + u and we cannot produce V + u +1. On the other hand, if u > V +1, 
then even by adding u to the collection we cannot produce V + 1. 

Another observation is that the order of the elements within the array makes no 
difference to the amounts that are constructible. However, by sorting the array allows 
us to stop when we reach a value that is too large to help, since all subsequent values 
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are at least as large as that value. Specifically, let M[i- 1] be the maximum constructible 
amount from the first i elements of the sorted array. If the next array element x is 
greater than M[i - 1] + 1, M[i - 1] is still the maximum constructible amount, so we 
stop and return M[i - 1] + 1 as the result. Otherwise, we set M[i] = M[i — 1] + x and 
continue with element (i + 1). 

To illustrate, suppose we are given (12,2,1,15,2,4). This sorts to (1,2,2,4,12,15). 
The maximum constructible amount we can make with the first element is 1. The 
second element, 2, is less than or equal to 1 + 1, so we can produce all values up to 
and including 3 now. The third element, 2, allows us to produce all values up to and 
including 5. The fourth element, 4, allows us to produce all values up to 9. The fifth 
element, 12 is greater than 9 +1, so we cannot produce 10. We stop—10 is the smallest 
number that cannot be constructed. 

The code implementing this approach is shown below. 

public static int smallestNonconstructibleValue(Listdnteger> A) { 

Collections.sort(A); 

int maxConstructibleValue = ©; 

for (int a : A) { 

if (a > maxConstructibleValue + 1) { 

break; 

} 

maxConstructibleValue += a; 

} 

return maxConstructibleValue + 1; 

} 


The time complexity as a function of n, the length of the array, is 0(n log n) to sort and 
0(n) to iterate, i.e., 0(n log n). 

The smallest nonconstructible change problem on the current page, the Towers of 
Hanoi 16.1 on Page 283 problem, the Levenshtein distance problem 17.2 on Page 309, 
and the maximum water that is trappable problem 25.35 on Page 501 are further 
examples of problems whose solution benefits from use of the concrete example 
pattern. 

Case analysis 

In case analysis, a problem is divided into a number of separate cases, and analyzing 
each such case individually suffices to solve the initial problem. Cases do not have to 
be mutually exclusive; however, they must be exhaustive, that is cover all possibilities. 
For example, to prove that for all n, n 3 mod 9 is 0,1, or 8, we can consider the cases 
n = 3m, n = 3m + 1, and n = 3m + 2. These cases are individually easy to prove, 
and are exhaustive. Case analysis is commonly used in mathematics and games of 
strategy. Here we consider an application of case analysis to algorithm design. 

Suppose you are given a set S of 25 distinct integers and a CPU that has a special 
instruction, SORT 5, that can sort five integers in one cycle. Your task is to identify the 
largest, second-largest, and third-largest integers in S using SORT 5 to compare and 
sort subsets of S; furthermore, you must minimize the number of calls to SORT 5. 

If all we had to compute was the largest integer in the set, the optimum approach 
would be to form five disjoint subsets Si,..., S 5 of S, sort each subset, and then sort 
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{maxS^.. .,maxS 5 }. This takes six calls to SORT5 but leaves ambiguity about the 
second and third largest integers. 

It may seem like many additional calls to SORT 5 are still needed. However, if you 
do a careful case analysis and eliminate all x e S for which there are at least three 
integers in S larger than x, only five integers remain and hence just one more call to 
SORT 5 is needed to compute the result. 

The solutions to Problems 25.1 on Page 435,25.4 on Page 439, and 14.6 on Page 244, 
among many others, also illustrate the case analysis pattern. 

Iterative refinement of a brute-force solution 

Many problems can be solved optimally by a simple algorithm that has a high time/s¬ 
pace complexity—this is sometimes referred to as a brute-force solution. Other terms 
are exhaustive search and generate-and-test. Often this algorithm can be refined to one 
that is faster. At the very least it may offer hints into the nature of the problem. 

As an example, suppose you were asked to write a program that takes an array A 
of n numbers, and rearranges A's elements to get a new array B having the property 
that B[ 0] < B[ 1] > B[ 2] < B[3] > B[ 4] < B[5] > . 

One straightforward solution is to sort A and interleave the bottom and top halves 
of the sorted array. Alternatively, we could sort A and then swap the elements 
at the pairs (A[1],A[2]),(A[3],A[4]),.... Both these approaches have the same time 
complexity as sorting, namely 0(n log n). 

You will soon realize that it is not necessary to sort A to achieve the desired 
configuration—you could simply rearrange the elements around the median, and 
then perform the interleaving. Median finding can be performed in time almost 
certain 0(ri), as per Solution 12.8 on Page 200, which is the overall time complexity of 
this approach. 

Finally, you may notice that the desired ordering is very local, and realize that it is 
not necessary to find the median. Iterating through the array and swapping A[i] and 
A[i + 1] when i is even and A[i] > A[i + 1] or i is odd and A[i] < A[i + 1] achieves the 
desired configuration. In code: 

public static void rearrange(Listdnteger> A) { 

for (int i = 1; i < A.sizeO; ++i) { 

if (((i % 2) == ® &<& A. get (i - 1) < A.get(i)) 

|| ((i % 2) != ® && A.get(i - 1) > A.get(i))) { 

Collections.swap(A, i - 1, i); 

} 

} 

} 


This approach has time complexity <9(n), which is the same as the approach based 
on median finding. However, it is much easier to implement and operates in an 
online fashion, i.e., it never needs to store more than two elements in memory or read 
a previous element. 

As another example of iterative refinement, consider the problem of string search: 
given two strings s (search string) and t (text), find all occurrences of s in t. Since s can 
occur at any offset in t, the brute-force solution is to test for a match at every offset. 
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This algorithm is perfectly correct; its time complexity is 0{nm) f where n and m are 
the lengths of s and t. See Solution 7.13 on Page 109 for details. 

After trying some examples you may see that there are several ways to improve 
the time complexity of the brute-force algorithm. As an example, if the character f [z] 
is not present in s you can skip past t[i]. Furthermore, this skipping works better 
if we match the search string from its end and work backwards. These refinements 
will make the algorithm very fast (linear time) on random text and search strings; 
however, the worst-case complexity remains 0(nm). 

You can make the additional observation that a partial match of s that does not 
result in a full match implies other offsets that cannot lead to full matches. If s = 
abdabcabc and if, starting backwards, we have a partial match up to abcabc that does 
not result in a full match, we know that the next possible matching offset has to be 
at least three positions ahead (where we can match the second abc from the partial 
match). 

By putting together these refinements you will have arrived at the famous Boyer- 
Moore string search algorithm—its worst-case time complexity is 0(n + m) (which 
is the best possible from a theoretical perspective); it is also one of the fastest string 
search algorithms in practice. 

As another example, the brute-force solution to computing the maximum subarray 
sum for an integer array of length n is to compute the sum of all subarrays, which 
has 0(n 3 ) time complexity. This can be improved to 0(n 2 ) by precomputing the 
sums of all the prefixes of the given arrays; this allows the sum of a subarray to be 
computed in 0(1) time. The natural divide-and-conquer algorithm has an 0(n log n) 
time complexity. Finally, one can observe that a maximum subarray must end at one 
of n indices, and the maximum subarray sum for a subarray ending at index i can be 
computed from previous maximum subarray sums, which leads to an 0(n) algorithm. 
Details are presented on Page 304. 

Many more sophisticated algorithms can be developed in this fashion. See Solu¬ 
tions 5.1 on Page 46, 25.4 on Page 439, 25.5 on Page 441, and 25.11 on Page 451 for 
examples. 

Reduction 

Consider the problem of determining if one string is a rotation of the other, e.g., 
"car" and "arc" are rotations of each other. A natural approach may be to rotate the 
first string by every possible offset and then compare it with the second string. This 
algorithm would have quadratic time complexity. 

You may notice that this problem is quite similar to string search, which can be 
done in linear time, albeit using a somewhat complex algorithm. Therefore, it is 
natural to try to reduce this problem to string search. Indeed, if we concatenate the 
second string with itself and search for the first string in the resulting string, we will 
find a match if and only if the two original strings are rotations of each other. This 
reduction yields a linear time algorithm for our problem. 

The reduction principle is also illustrated in the solutions to converting between 
different base representations (Problem 7.2 on Page 96), computing all permutations 


34 



(Problem 16.3 on Page 287), and finding the minimum number of pictures needed to 
photograph a set of teams (Problem 19.8 on Page 369). 

Usually, you try to reduce the given problem to an easier problem. Sometimes, 
however, you need to reduce a problem known to be difficult to the given prob¬ 
lem. This shows that the given problem is difficult, which justifies heuristics and 
approximate solutions. 

Graph modeling 

Drawing pictures is a great way to brainstorm for a potential solution. If the relation¬ 
ships in a given problem can be represented using a graph, quite often the problem 
can be reduced to a well-known graph problem. For example, suppose you are given 
a set of exchange rates among currencies and you want to determine if an arbitrage 
exists, i.e., there is a way by which you can start with one unit of some currency C 
and perform a series of barters which results in having more than one unit of C. 

Table 4.4 shows a representative example. An arbitrage is possible for this 
set of exchange rates: 1 USD -> 1 x 0.8123 = 0.8123 EUR -> 0.8123 x 1.2010 = 
0.9755723 CHF -> 0.9755723 X 80.39 = 78.426257197 JPY -> 78.426257197 x 0.0128 = 
1.00385609212 USD. 

Table 4.4: Exchange rates for seven major currencies. 


Symbol 

USD 

EUR 

GBP 

JPY 

CHF 

CAD 

AUD 

USD 

1 

0.8123 

0.6404 

78.125 

0.9784 

0.9924 

0.9465 

EUR 

1.2275 

1 

0.7860 

96.55 

1.2010 

1.2182 

1.1616 

GBP 

1.5617 

1.2724 

1 

122.83 

1.5280 

1.5498 

1.4778 

JPY 

0.0128 

0.0104 

0.0081 

1 

1.2442 

0.0126 

0.0120 

CHF 

1.0219 

0.8327 

0.6546 

80.39 

1 

1.0142 

0.9672 

CAD 

1.0076 

0.8206 

0.6453 

79.26 

0.9859 

1 

0.9535 

AUD 

1.0567 

0.8609 

0.6767 

83.12 

1.0339 

1.0487 

1 


We can model the problem with a graph where currencies correspond to vertices, 
exchanges correspond to edges, and the edge weight is set to the logarithm of the 
exchange rate. If we can find a cycle in the graph with a positive weight, we would 
have found such a series of exchanges. Such a cycle can be solved using the Bellman- 
Ford algorithm. This is described in Solution 25.40 on Page 511. The solutions 
to the problems of painting a Boolean matrix (Problem 19.2 on Page 357), string 
transformation (Problem 19.7 on Page 366), and wiring a circuit (Problem 19.6 on 
Page 364) also illustrate modeling with graphs. 

Sorting 

Certain problems become easier to understand, as well as solve, when the input is 
sorted. The solution to the calendar rendering problem (Problem 14.4 on Page 240) 
entails taking a set of intervals and computing the maximum number of intervals 
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whose intersection is nonempty. Naive strategies yield quadratic run times. However, 
once the interval endpoints have been sorted, it is easy to see that a point of maximum 
overlap can be determined by a linear time iteration through the endpoints. 

Often it is not obvious what to sort on—for example, we could have sorted the 
intervals on starting points rather than endpoints. This sort sequence, which in some 
respects is more natural, does not work. However, some experimentation with it will, 
in all likelihood, lead to the correct criterion. 

Sorting is not appropriate when an 0(n) (or better) algorithm is possible. For 
example, the A:th largest element in an array can be computed in almost certain 0(n) 
time (Solution 12.8 on Page 200). Another good example of a problem where a total 
ordering is not required is the problem of rearranging elements in an array described 
on Page 33. Furthermore, sorting can obfuscate the problem. For example, given an 
array A of numbers, if we are to determine the maximum of A[i\ - A[j], for i < j, 
sorting destroys the order and complicates the problem. 

Recursion 

A recursive function consists of base cases and calls to the same function with different 
arguments. A recursive algorithm is often appropriate when the input is expressed 
using recursive rules, such as a computer grammar. More generally, searching, 
enumeration, divide-and-conquer, and decomposing a complex problem into a set of 
similar smaller instances are all scenarios where recursion may be suitable. 

String matching exemplifies the use of recursion. Suppose you were asked to 
write a Boolean-valued function which takes a string and a matching expression, and 
returns true if and only if the matching expression "matches" the string. Specifically, 
the matching expression is itself a string, and could be 

• x, where x is a character, for simplicity assumed to be a lowercase letter (matches 
the string "x"). 

• . (matches any string of length 1). 

• x* (matches the string consisting of zero or more occurrences of the character x). 

• .* (matches the string consisting of zero or more of any characters). 

• f*i ? 2 / where r\ and r 2 are regular expressions of the given form (matches any 
string that is the concatenation of strings Si and S 2 , where r\ matches si and r 2 
matches s 2 ). 

This problem can be solved by checking a number of cases based on the first one 
or two characters of the matching expression, and recursively matching the rest of 
the string. Details are given in Solution 25.26 on Page 479. 

Divide-and-conquer 

A divide-and-conquer algorithm works by decomposing a problem into two or more 
smaller independent subproblems until it gets to instances that are simple enough to 
be solved directly; the results from the subproblems are then combined. More details 
and examples are given in Chapter 18; we illustrate the basic idea below. 

A triomino is formed by joining three unit-sized squares in an L-shape. A mutilated 
chessboard (henceforth 8x8 Mboard) is made up of 64 unit-sized squares arranged 
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in an 8 X 8 square, minus the top-left square, as depicted in Figure 4.2(a). Suppose 
you are asked to design an algorithm that computes a placement of 21 triominoes that 
covers the 8x8 Mboard. Since the 8x8 Mboard contains 63 squares, and we have 
21 triominoes, a valid placement cannot have overlapping triominoes or triominoes 
which extend out of the 8x8 Mboard. 



Figure 4.2: Mutilated chessboards. 


Divide-and-conquer is a good strategy for this problem. Instead of the 8x8 
Mboard, let's consider an nxn Mboard. A 2 X 2 Mboard can be covered with one 
triomino since it is of the same exact shape. You may hypothesize that a triomino 
placement for an nx n Mboard with the top-left square missing can be used to compute 
a placement for an (n + 1) X (n + 1) Mboard. However, you will quickly see that this 
line of reasoning does not lead you anywhere. 

Another hypothesis is that if a placement exists for an nxn Mboard, then one also 
exists for a 2n x 2n Mboard. Now we can apply divide-and-conquer. Take four nxn 
Mboards and arrange them to form a 2n X 2n square in such a way that three of the 
Mboards have their missing square set towards the center and one Mboard has its 
missing square outward to coincide with the missing corner of a 2 nx 2 n Mboard, as 
shown in Figure 4.2(b). The gap in the center can be covered with a triomino and, by 
hypothesis, we can cover the four n X n Mboards with triominoes as well. Hence, a 
placement exists for any n that is a power of 2. In particular, a placement exists for 
the 2 3 X 2 3 Mboard; the recursion used in the proof directly yields the placement. 

Divide-and-conquer is usually implemented using recursion. However, the two 
concepts are not synonymous. Recursion is more general—subproblems do not have 
to be of the same form. 

In addition to divide-and-conquer, we used the generalization principle above. 
The idea behind generalization is to find a problem that subsumes the given problem 
and is easier to solve. We used it to go from the 8x8 Mboard to the 2" X 2" Mboard. 

Another good example of divide-and-conquer is computing the number of pairs 
of elements in an array that are out of sorted order (Problem 25.28 on Page 483). 
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Dynamic programming 


Dynamic programming (DP) is applicable when the problem has the "optimal sub¬ 
structure" property, that is, it is possible to reconstruct a solution to the given instance 
from solutions to subinstances of smaller problems of the same kind. A key aspect 
of DP is maintaining a cache of solutions to subinstances. DP can be implemented 
recursively (in which case the cache is typically a dynamic data structure such as a 
hash table or a BST), or iteratively (in which case the cache is usually a one- or multi¬ 
dimensional array). It is most natural to design a DP algorithm using recursion. 
Usually, but not always, it is more efficient to implement it using iteration. 

As an example of the power of DP, consider the problem of determining the 
number of combinations of 2, 3, and 7 point plays that can generate a score of 
222. Let C(s) be the number of combinations that can generate a score of s. Then 
C(222) = C(222 - 7) + C(222 - 3) + C(222 - 2), since a combination ending with a 2 point 
play is different from the one ending with a 3 point play, and a combination ending 
with a 3 point play is different from the one ending with a 7 point play, etc. 

The recursion ends at small scores, specifically, when (1.) s < 0 => C(s) = 0, and 
(2.) s = 0 => C(s) = 1. 

Implementing the recursion naively results in multiple calls to the same subin¬ 
stance. Let C(a) C(b) indicate that a call to C with input a directly calls C with input 
b . Then C(222) will be called in the order C(222) C(222 - 7) ^ C((222 - 7) - 2), as 

well as C(222) -+ C(222 - 3) -+ C((222 - 3) - 3) ^ C(((222 - 3) - 3) - 3). 

This phenomenon results in the run time increasing exponentially with the size 
of the input. The solution is to store previously computed values of C in an array of 
length 223. Details are given in Solution 17.1 on Page 306. 

Dynamic programming is the subject of Chapter 17. The solutions to problems 17.2 
on Page 309, 17.3 on Page 312, and 17.6 on Page 317 are good illustrations of the 
principles of dynamic programming. 

Greedy algorithms 

A greedy algorithm is one which makes decisions that are locally optimum and 
never changes them. This strategy does not always yield the optimum solution. 
Furthermore, there may be multiple greedy algorithms for a given problem, and only 
some of them are optimum. 

For example, consider 2 n cities on a line, half of which are white, and the other 
half are black. We want to map white to black cities in a one-to-one fashion so that 
the total length of the road sections required to connect paired cities is minimized. 
Multiple pairs of cities may share a single section of road, e.g., if we have the pairing 
(0,4) and (1,2) then the section of road between Cities 0 and 4 can be used by Cities 1 
and 2. 

The most straightforward greedy algorithm for this problem is to scan through the 
white cities, and, for each white city, pair it with the closest unpaired black city. This 
algorithm leads to suboptimum results. Consider the case where white cities are at 
0 and at 3 and black cities are at 2 and at 5. If the straightforward greedy algorithm 
processes the white city at 3 first, it pairs it with 2, forcing the cities at 0 and 5 to pair 
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up, leading to a road length of 5, whereas the pairing of cities at 0 and 2, and 3 and 5 
leads to a road length of 4. 

However, a slightly more sophisticated greedy algorithm does lead to optimum 
results: iterate through all the cities in left-to-right order, pairing each city with the 
nearest unpaired city of opposite color. More succinctly, let W and B be the arrays of 
white and black city coordinates. Sort W and B, and pair W[i] with B[i]. We can prove 
this leads to an optimum pairing by induction. The idea is that the pairing for the 
first city must be optimum, since if it were to be paired with any other city, we could 
always change its pairing to be with the nearest black city without adding any road. 

Chapter 18 contains a number of problems whose solutions employ greedy al¬ 
gorithms. The solutions to Problems 25.34 on Page 497 and 18.2 on Page 335 are 
especially representative. Several problems in other chapters also use a greedy algo¬ 
rithm as a key subroutine. 

Invariants 

One common approach to designing an efficient algorithm is to use invariants. Briefly, 
an invariant is a condition that is true during execution of a program. This condition 
may be on the values of the variables of the program, or on the control logic. A 
well-chosen invariant can be used to rule out potential solutions that are suboptimal 
or dominated by other solutions. 

An invariant can also be used to analyze a given algorithm, e.g., to prove its 
correctness, or analyze its time complexity. Here our focus is on designing algorithms 
with invariants, not analyzing them. 

As an example, consider the 2-sum problem. We are given an array A of sorted 
integers, and a target value K. We want to know if there exist entries i and j in A such 
that A[i]+A[j] = K. 

The brute-force algorithm for the 2-sum problem consists of a pair of nested for 
loops. Its complexity is 0{n 2 ), where n is the length of A. A faster approach is to add 
each element of A to a hash H, and test for each i if K - A[i] is present in H. While 
reducing time complexity to 0(n), this approach requires 0(n) additional storage for 
H. 

We want to compute i and j such that A[i] + A[j] = K. Without loss of generality, 
we can take i < j. We know that 0 < i, and j < n — 1. A natural approach then is 
to initialize i to 0, and j to n - 1, and then update i and j preserving the following 
invariant: 

• no i' < i can ever be paired with any f such that A[i f ] + A[j'] = K, and 

• no f > j can ever be paired with any i' such that A[i f ] + A[j f ] = K. 

The invariant is certainly true at initialization, since there are no i' < 0 and j' > n - 1. 
To show how i and j can be updated while ensuring the invariant continues to hold, 
consider A[i] + A[j\. If A[i] + A[j] = K, we are done. Otherwise, consider the case 
A[i\ + A[j] < K. We know from the invariant that for no j' > j is there a solution 
in which the element with the larger index is j '. The element at i cannot be paired 
with any element at an index f smaller than j —because A is sorted, A[i] + A[j'] < 
A[i] + A[j] < K. Therefore, we can increment i, and preserve the invariant. Similarly, 
in the case A[i\ + A[j] > K, we can decrement j and preserve the invariant. 
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We terminate when either A [i] + A[j] = K (success) or i > j (failure). At each step, 
we increment or decrement i or j. Since there are at most n steps, and each takes (9(1) 
time, the time complexity is 0(n). Correctness follows from the fact that the invariant 
never discards a value for i or j which could possibly be the index of an element 
which sums with another element to K. 

Identifying the right invariant is an art. Usually, it is arrived at by studying 
concrete examples and then making an educated guess. Often the first invariant is 
too strong, i.e., it does not hold as the program executes, or too weak, i.e., it holds 
throughout the program execution but cannot be used to infer the result. 

The solutions to Problems 18.4 on Page 340 and 25.36 on Page 502 make use of 
invariants to solve generalizations of the 2-sum problem. Binary search, which we 
study in Chapter 12, uses an invariant to design an (9(log n) algorithm for searching in 
a sorted array. Solution 12.5 on Page 195, which uses elimination in conjunction with 
an invariant to compute the square root of a real number, is especially instructive. 
Other examples of problems whose solutions make use of invariants are finding 
the longest subarray containing all entries (Problem 13.9 on Page 224), enumerating 
numbers of the form a + b V2 (Problem 15.7 on Page 267), and finding the majority 
element (Problem 18.5 on Page 341). 

4.3 Complexity Analysis 

The run time of an algorithm depends on the size of its input. A common approach to 
capture the run time dependency is by expressing asymptotic bounds on the worst- 
case run time as a function of the input size. Specifically, the run time of an algorithm 
on an input of size n is O ( f(n )) if, for sufficiently large n, the run time is not more 
than f(n) times a constant. The big-<9 notation indicates an upper bound on running 
time. 

As an example, searching an unsorted array of integers of length n, for a given 
integer, has an asymptotic complexity of 0(n) since in the worst-case, the given integer 
may not be present. Similarly, consider the naive algorithm for testing primality that 
tries all numbers from 2 to the square root of the input number n. What is its 
complexity? In the best-case, n is divisible by 2. However, in the worst-case, the 
input may be a prime, so the algorithm performs yfn iterations. 

Complexity theory is applied in a similar manner when analyzing the space re¬ 
quirements of an algorithm. The space needed to read in an instance is not included; 
otherwise, every algorithm would have 0(n) space complexity. Several of our prob¬ 
lems call for an algorithm that uses 0(1) space. Specifically, it should be possible 
to implement the algorithm without dynamic memory allocation (explicitly, or in¬ 
directly, e.g., through library routines). Furthermore, the maximum depth of the 
function call stack should also be a constant, independent of the input. The stan¬ 
dard algorithm for depth-first search of a graph is an example of an algorithm that 
does not perform any dynamic allocation, but uses the function call stack for implicit 
storage—its space complexity is not 0(1). 

A streaming algorithm is one in which the input is presented as a sequence of 
items and is examined in only a few passes (typically just one). These algorithms 
have limited memory available to them (much less than the input size) and also 


40 



limited processing time per item. Algorithms for computing summary statistics on 
log file data often fall into this category. 

Many authors, ourselves included, will refer to the time complexity of an algorithm 
as its complexity without the time qualification. The space complexity is always 
qualified as such. 

4.4 Intractability 

In real-world settings you will often encounter problems that can be solved using 
efficient algorithms such as binary search and shortest paths. As we will see in the 
coming chapters, it is often difficult to identify such problems because the algorithmic 
core is obscured by details. Sometimes, you may encounter problems which can be 
transformed into equivalent problems that have an efficient textbook algorithm, or 
problems that can be solved efficiently using meta-algorithms such as DP. 

Occasionally, the problem you are given is intractable—i.e., there may not exist 
an efficient algorithm for the problem. Complexity theory addresses these problems. 
Some have been proved to not have an efficient solution but the vast majority are only 
conjectured to be intractable. The conjunctive normal form satisfiability (CNF-SAT) 
problem is an example of a problem that is conjectured to be intractable. Specifically, 
the CNF-SAT problem belongs to the complexity class NP—problems for which a 
candidate solution can be efficiently checked—and is conjectured to be the hardest 
problem in this class. 

When faced with a problem P that appears to be intractable, the first thing to do is 
to prove intractability. This is usually done by taking a problem which is known to be 
intractable and showing how it can be efficiently reduced to P. Often this reduction 
gives insight into the cause of intractability. 

Unless you are a complexity theorist, proving a problem to be intractable is only 
the starting point. Remember something is a problem only if it has a solution. There 
are a number of approaches to solving intractable problems: 

• brute-force solutions, including dynamic programming, which have exponen¬ 
tial time complexity, may be acceptable, if the instances encountered are small, 
or if the specific parameter that the complexity is exponential in is small; 

• search algorithms, such as backtracking, branch-and-bound, and hill-climbing, 
which prune much of the complexity of a brute-force search; 

• approximation algorithms which return a solution that is provably close to 
optimum; 

• heuristics based on insight, common case analysis, and careful tuning that may 
solve the problem reasonably well; 

• parallel algorithms, wherein a large number of computers can work on subparts 
simultaneously. 

Solutions 19.7 on Page 366,13.13 on Page 230,16.9 on Page 296,25.27 on Page 481,17.6 
on Page 317, 25.30 on Page 488, and 21.9 on Page 398 illustrate the use of some of 
these techniques. 

Don't forget it may be possible to dramatically change the problem formulation 
while still achieving the higher level goal, as illustrated in Figure 4.3 on the next page. 
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Figure 4.3: Traveling Salesman Problem by xkcd. 
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Part II 

Problems 



Chapter 

I™ 


Representation is the essence of programming. 

— "The Mythical Man Month," 
F. P. Brooks, 1975 


A program updates variables in memory according to its instructions. Variables 
come in types—a type is a classification of data that spells out possible values for 
that type and the operations that can be performed on it. A type can be provided 
by the language or defined by the programmer. Many languages provide types for 
Boolean, integer, character and floating point data. Often, there are multiple integer 
and floating point types, depending on signedness and precision. The width of these 
types is the number of bits of storage a corresponding variable takes in memory. For 
example, most implementations of C++ use 32 or 64 bits for an int. In Java an int is 
always 32 bits. 

Primitive types boot camp 

Writing a program to count the number of bits that are set to 1 in an integer is a good 
way to get up to speed with primitive types. The following program tests bits one- 
at-a-time starting with the least-significant bit. It illustrates shifting and masking; it 
also shows how to avoid hard-coding the size of the integer word. 

public static short countBits (int x) { 
short numBits = ®; 
while (x != ®) { 

numBits += (x & 1) ; 
x »>= 1; 

} 

return numBits; 

} 

Since we perform 0(1) computation per bit, the time complexity is 0(ri), where n is 
the number of bits in the integer word. Note that, by definition, the time complexity 
is for the worst case input, which for this example is the word (111... 11 ) 2 . In the best 
case, the time complexity is 0(1), e.g., if the input is 0. The techniques in Solution 5.1 
on the facing page, which is concerned with counting the number of bits modulo 2, 
i.e., the parity, can be used to improve the performance of the program given above. 

Know your primitive types 

You should know the primitive types very intimately, e.g., sizes, ranges, signedness 
properties, and operators. You should also know the utility methods for primi¬ 
tive types in Math. It's important to understand the difference between box-types 
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Be very comfortable with the bitwise operators, particularly XOR. [Problem 5.2] 

Understand how to use masks and create them in an machine independent way 
[Problem 5.6] 

Know fast ways to clear the lowermost set bit (and by extension, set the lowermost 
0, get its index, etc.) [Problem 5.1] 

Understand signedness and its implications to shifting. [Problem 5.5] 

Consider using a cache to accelerate operations by using it to brute-force small 
inputs. [Problem 5.3] 

Be aware that commutativity and associativity can be used to perform operations 
in parallel and reorder operations. [Problem 5.1] 


(Integer, Double, etc.) and primitive types, and the role of auto-boxing, and the lim¬ 
its of auto-boxing. The Random library is also very helpful, especially when writing 
tests. 

• Be very familiar with the bit-wise operators such as 6&4,11 2, 8»1, -16»>2, 
1«1Q, ~<S>, 15~x. 

• Know the constants denoting the maximum and minimum values for 
numeric types, e.g., Integer.MIN_VALUE, Float.MAX_VALUE, Double.SIZE, 
Boolean.TRUE. 

• Know the box-types, especially the factories, e.g., Double .valueOf(“l. 23”), 
Boolean.valueOf(true),Integer.parselnt(“42”),Float.toString(-1.23). 

• Prefer the box-type static methods for comparing values, e.g.. 

Double .compare(x, 1.23) == 0 rather than x == 1.23—these methods are 
far more resilient to floating point values like infinity, negative infinity, NaN. 

• The key methods in Math are abs(-34.5), ceil(2.17), floor(3.14), 
min(x,-4), max(3.14, y),pow(2.71, 3.14),and sqrt(225). 

• Understand the limits of autoboxing, e.g., why Character [] C = new 
char [] { ’ a ’ , ’ b ’}; will not compile. 

• Know how to interconvert integers, characters, and strings, e.g.. 

Character.getNumericValue(x) (or just x - ’O’) to convert a digit char¬ 
acter to an integer. String .value0f( 123) to convert an integer to a string, 
etc. 

• The key methods in Random are nextlnt(16), nextlnt(), nextBooleanO, and 
nextDoubleO (which returns a value in [0,1)). 

5.1 Computing the parity of a word 

The parity of a binary word is 1 if the number of Is in the word is odd; otherwise, 
it is 0. For example, the parity of 1011 is 1, and the parity of 10001000 is 0. Parity 
checks are used to detect single bit errors in data storage and communication. It is 
fairly straightforward to write code that computes the parity of a single 64-bit word. 
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How would you compute the parity of a very large number of 64-bit words? 

Hint: Use a lookup table, but don't use 2 64 entries! 

Solution: The brute-force algorithm iteratively tests the value of each bit while track¬ 
ing the number of Is seen so far. Since we only care if the number of Is is even or 
odd, we can store the number modulo 2. 

public static short parity(long x) { 
short result = ®; 
while (x != ®) { 
result A = (x & 1); 
x »>= 1; 

} 

return result; 


The time complexity is 0(n), where n is the word size. 

On Page 25 we showed how to erase the lowest set bit in a word in a single 
operation. This can be used to improve performance in the best- and average-cases. 

public static short parity (long x) { 
short result = ®; 
while (x != ®) { 
result A = 1; 

x &= (x - 1); // Drops the lowest set bit of x. 

} 

return result; 


Let k be the number of bits set to 1 in a particular word. (For example, for 10001010, 
k- 3.) Then time complexity of the algorithm above is 0(k). 

The problem statement refers to computing the parity for a very large number 
of words. When you have to perform a large number of parity computations, and, 
more generally, any kind of bit fiddling computations, two keys to performance are 
processing multiple bits at a time and caching results in an array-based lookup table. 

First we demonstrate caching. Clearly, we cannot cache the parity of every 64-bit 
integer—we would need 2 64 bits of storage, which is of the order of ten trillion ex¬ 
abytes. However, when computing the parity of a collection of bits, it does not matter 
how we group those bits, i.e., the computation is associative. Therefore, we can com¬ 
pute the parity of a 64-bit integer by grouping its bits into four nonoverlapping 16 bit 
subwords, computing the parity of each subwords, and then computing the parity of 
these four subresults. We choose 16 since 2 16 = 65536 is relatively small, which makes 
it feasible to cache the parity of all 16-bit words using an array. Furthermore, since 
16 evenly divides 64, the code is simpler than if we were, for example, to use 10 bit 
subwords. 

We illustrate the approach with a lookup table for 2-bit words. The cache is 
(0,1,1,0)—these are the parities of (00), (01), (10), (11), respectively. To compute the 
parity of (11001010) we would compute the parities of (11), (00), (10), (10). By table 
lookup we see these are 0,0,1,1, respectively, so the final result is the parity of 0,0,1,1 
which is 0. 
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To lookup the parity of the first two bits in (11101010), we right shift by 6, to get 
(00000011), and use this as an index into the cache. To lookup the parity of the next 
two bits, i.e., (10), we right shift by 4, to get (10) in the two least-significant bit places. 
The right shift does not remove the leading (11)—it results in (00001110). We cannot 
index the cache with this, it leads to an out-of-bounds access. To get the last two 
bits after the right shift by 4, we bitwise-AND (00001110) with (00000011) (this is the 
"mask" used to extract the last 2 bits). The result is (00000010). Similar masking is 
needed for the two other 2-bit lookups. 

public static short parity(long x) { 
final int W0RD_SIZE = 16; 
final int BIT.MASK = QxFFFF; 
return (short) ( 

precomputedParity [(int) ((x »> (3 * WORD.SIZE)) & BIT_MASK)] 

A precomputedParity [(int) ((x »> (2 * WORD.SIZE)) & BIT_MASK)] 

A precomputedParity [(int) ((x »> W0RD_SIZE) & BIT_MASK)] 

A precomputedParity [(int) (x & BIT_MASK)]); 

} 


The time complexity is a function of the size of the keys used to index the lookup 
table. Let L be the width of the words for which we cache the results, and n the 
word size. Since there are n/L terms, the time complexity is Q(n/L), assuming word- 
level operations, such as shifting, take 0( 1) time. (This does not include the time for 
initialization of the lookup table.) 

The XOR of two bits is 0 if both bits are 0 or both bits are 1; otherwise it is 1. XOR has 
the property of being associative (as previously described), as well as commutative, 
i.e., the order in which we perform the XORs does not change the result. The XOR of 
a group of bits is its parity. We can exploit this fact to use the CPU's word-level XOR 
instruction to process multiple bits at a time. 

For example, the parity of (^ 63 /^ 62 /-. .,& 3 / b 2 ,b\,bo) equals the parity of the XOR 
of (b 6 3 , b 6 2 ,..., b 32 ) and (b 3 i, b 30 ,..., b 0 ). The XOR of these two 32-bit values can be 
computed with a single shift and a single 32-bit XOR instruction. We repeat the same 
operation on 32-, 16-, 8-, 4-, 2-, and 1-bit operands to get the final result. Note that the 
leading bits are not meaningful, and we have to explicitly extract the result from the 
least-significant bit. 

We illustrate the approach with an 8-bit word. The parity of (11010111) is the same 
as the parity of (1101) XORed with (0111), i.e., of (1010). This in turn is the same 
as the parity of (10) XORed with (10), i.e., of (00). The final result is the XOR of (0) 
with (0), i.e., 0. Note that the first XOR yields (11011010), and only the last 4 bits are 
relevant going forward. The second XOR yields (11101100), and only the last 2 bits 
are relevant. The third XOR yields (10011010). The last bit is the result, and to extract 
it we have to bitwise-AND with (00000001). 

public static short parity(long x) { 

x A = x »> 32; 
x A = x »> 16; 
x A = x »> 8; 
x A = x »> 4; 
x A = x »> 2; 
x A = x »> 1; 
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return (short)(x & ®xl); 


} 


The time complexity is <9(log n), where n is the word size. 

Note that we could have combined caching with word-level operations, e.g., by 
doing a lookup once we get to 16 bits. The actual runtimes depend on the input 
data, e.g., the refinement of the brute-force algorithm is very fast on sparse inputs. 
However, for random inputs, the refinement of the brute-force is roughly 20% faster 
than the brute-force algorithm. The table-based approach is four times faster still, 
and using associativity reduces run time by another factor of two. 


5.2 Swap bits 

There are a number of ways in which bit manipulations can be accelerated. For 
example, as described on Page 25, the expression x & (x - 1) clears the lowest set 
bit in x, and x & ~(x - 1) extracts the lowest set bit of x. Here are a few examples: 
16&(16-1) = 0,11&(11-1) = 10,20&(20-l) = 16,16&~(16-1) = 16,11&~(11-1) = 1, 
and 20&~(20 - 1) = 4. 


0 

0 

0 

0 

0 

0 

0 

1 


MSB LSB 

(a) The 8-bit integer 73 can be viewed as array of 
bits, with the LSB being at index 0. 
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(b) The result of swapping the bits at indices 1 and 
6, with the LSB being at index 0. The corresponding 
integer is 11. 


Figure 5.1: Example of swapping a pair of bits. 


A 64-bit integer can be viewed as an array of 64 bits, with the bit at index 0 corre¬ 
sponding to the least significant bit (LSB), and the bit at index 63 corresponding to 
the most significant bit (MSB). Implement code that takes as input a 64-bit integer 
and swaps the bits at indices i and j. Figure 5.1 illustrates bit swapping for an 8-bit 
integer. 

Hint: When is the swap necessary? 

Solution: A brute-force approach would be to use bitmasks to extract the ith and 
yth bits, saving them to local variables. Consequently, write the saved yth bit to 
index i and the saved zth bit to index j, using a combination of bitmasks and bitwise 
operations. 

The brute-force approach works generally, e.g., if we were swapping objects stored 
in an array. However, since a bit can only take two values, we can do a little better. 
Specifically, we first test if the bits to be swapped differ. If they do not, the swap 
does not change the integer. If the bits are different, swapping them is the same as 
flipping their individual values. For example in Figure 5.1, since the bits at Index 1 
and Index 6 differ, flipping each bit has the effect of a swap. 
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In the code below we use standard bit-fiddling idioms for testing and flipping bits. 
Overall, the resulting code is slightly more succinct and efficient than the brute force 
approach. 

public static long swapBits (long x, int i, int j) { 

// Extract the i-th and j-th bits, and see if they differ. 
if CC(x »> i) & 1) ! = C(x »> j) & 1)) { 

// i-th and j-th bits differ. We will swap them by flipping their values. 

// Select the bits to flip with bitMask. Since x A l = SI when x = 1 and 1 
// when x = SI, we can perform the flip XOR. 
long bitMask = (1L « i) | (1L « j); 
x A = bitMask; 

} 

return x; 


The time complexity is 0(1), independent of the word size. 


5.3 Reverse bits 

Write a program that takes a 64-bit word and returns the 64-bit word consisting of 
the bits of the input word in reverse order. For example, if the input is alternating Is 
and Os, i.e., (1010... 10), the output should be alternating Os and Is, i.e., (0101... 01). 

Hint: Use a lookup table. 

Solution: If we need to perform this operation just once, there is a simple brute- 
force algorithm: iterate through the 32 least significant bits of the input, and swap 
each with the corresponding most significant bit, using, for example, the approach in 
Solution 5.2 on the preceding page. 

To implement reverse when the operation is to be performed repeatedly, we look 
more carefully at the structure of the input, with an eye towards using a cache. 
Let the input consist of the four 16-bit words yz,y 2 ,y\,y§, with y 3 holding the most 
significant bits. Then the 16 least significant bits in the reverse come from y 3 . To be 
precise, these bits appear in the reverse order in which they do in y 3 . For example, if 
y 3 is (1110000000000001), then the 16 LSBs of the result are (1000000000000111). 

Similar to computing parity (Problem 5.1 on Page 45), a very fast way to reverse 
bits for 16-bit words when we are performing many reverses is to build an array- 
based lookup-table A such that for every 16-bit number y, A[y] holds the bit-reversal 
of y. We can then form the reverse of x with the reverse of yo in the most significant 
bit positions, followed by the reverse of yi, followed by the reverse of y 2 , followed by 
the reverse of y 3 . 

We illustrate the approach with 8-bit words and 2-bit lookup table keys. The 
table is rev = ((00), (10), (01), (11)). If the input is (10010011), its reverse is 
rev(ll),rev(00),rev(01),rev(10), i.e., (11001001). 

public static long reverseBits (long x) { 
final int W0RD_SIZE = 16; 
final int BIT.MASK = ®xFFFF; 

return precomputedReverse[ (int) (x & BIT_MASK)] « (3 * W0RD_SIZE) 

| precomputedReverse [(int) ((x »> W0RD.SIZE) & BIT_MASK)] 
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« (2 * WORD.SIZE) 

| precomputedReverse [(int) ((x »> (2 * WORD.SIZE)) & BIT_MASK)] 
« WORD.SIZE 

| precomputedReverse [(int) C(x »> (3 * WORD.SIZE)) & BIT_MASK)]; 


The time complexity is identical to that for Solution 5.1 on Page 46, i.e., 0(n/L), for 
n-bit integers and L-bit cache keys. 

5.4 Find a closest integer with the same weight 

Define the weight of a nonnegative integer x to be the number of bits that are set to 
1 in its binary representation. For example, since 92 in base-2 equals (1011100)2, the 
weight of 92 is 4. 

Write a program which takes as input a nonnegative integer x and returns a number 
y which is not equal to x, but has the same weight as x and their difference, \y - x\, is 
as small as possible. You can assume x is not 0, or all Is. For example, if x = 6, you 
should return 5. 

Hint: Start with the least significant bit. 

Solution: A brute-force approach might be to try all integers x -1, x +1, x - 2, x + 2,..., 
stopping as soon as we encounter one with the same weight at x. This performs very 
poorly on some inputs. One way to see this is to consider the case where x = 2 3 = 8. 
The only numbers with a weight of 1 are powers of 2. Thus, the algorithm will try 
the following sequence: 7,9,6,10,5,11,4, stopping at 4 (since its weight is the same 
as 8's weight). The algorithm tries 2 3_1 numbers smaller than 8, namely, 7,6,5,4, and 
2 3_1 - 1 numbers greater than 8, namely, 9,10,11. This example generalizes. Suppose 
x — 2 30 . The power of 2 nearest to 2 30 is 2 29 . Therefore this computation will evaluate 
the weight of all integers between 2 30 and 2 29 and between 2 30 and 2 30 + 2 29 - 1, i.e., 
over one billion integers. 

Heuristically, it is natural to focus on the LSB of the input, specifically, to swap 
the LSB with rightmost bit that differs from it. This yields the correct result for 
some inputs, e.g., for (10) 2 it returns (01) 2 , which is the closest possible. However, 
more experimentation shows this heuristic does not work generally. For example, for 
(111) 2 (7 in decimal) it returns (1110) 2 which is 14 in decimal; however, (1011) 2 (11 in 
decimal) has the same weight, and is closer to (111) 2 . 

A little math leads to the correct approach. Suppose we flip the bit at index kl and 
flip the bit at index k2, kl > kl. Then the absolute value of the difference between the 
original integer and the new one is 2 kl - 2 k2 . To minimize this, we should make kl as 
small as possible and k2 as close to kl. 

Since we must preserve the weight, the bit at index kl has to be different from the 
bit in kl, otherwise the flips lead to an integer with different weight. This means the 
smallest kl can be is the rightmost bit that's different from the LSB, and kl must be 
the very next bit. In summary, the correct approach is to swap the two rightmost 
consecutive bits that differ. 

static final int NUM_UNSIGN_BITS = 63; 
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public static long closestlntSameBitCount (long x) { 

// x is assumed to be non-negative so we know the leading bit is (9. life 
// restrict to our attention to 63 LSBs. 
for (int i = ®; i < NUM_UNSIGN_BITS - 1; ++i) { 
if C(((x »> i) & 1) ! = ((x »> (i + 1)) & 1))) { 

x A = (1L « i) | (1L « (i + 1)); // Swaps bit-i and bit-(i + 1 ). 
return x; 

} 

} 

// Throw error if all bits of x are <9 or 1. 

throw new IllegalArgumentExceptionC'All bits are ® or 1"); 


The time complexity is 0(ri ), where n is the integer width. 
Variant: Solve the same problem in (9(1) time and space. 


5.5 Compute x X y without arithmetical operators 

Sometimes the processors used in ultra low-power devices such as hearing aids do 
not have dedicated hardware for performing multiplication. A program that needs 
to perform multiplication must do so explicitly using lower-level primitives. 

Write a program that multiplies two nonnegative integers. The only operators you 
are allowed to use are 

• assignment, 

• the bitwise operators », «, |, &, * and 

• equality checks and Boolean combinations thereof. 

You may use loops and functions that you write yourself. These constraints imply, 
for example, that you cannot use increment or decrement, or test if x < y. 

Hint: Add using bitwise operations; multiply using shift-and-add. 

Solution: A brute-force approach would be to perform repeated addition, i.e., initial¬ 
ize the result to 0 and then add x to it y times. For example, to form 5x3, we would 
start with 0 and repeatedly add 5, i.e., form 0 + 5,5 + 5,10 + 5. The time complexity 
is very high—as much as 0(2 n ), where n is the number of bits in the input, and it 
still leaves open the problem of adding numbers without the presence of an add 
instruction. 

The algorithm taught in grade-school for decimal multiplication does not use 
repeated addition—it uses shift and add to achieve a much better time complexity. 
We can do the same with binary numbers—to multiply x and y we initialize the result 
to 0 and iterate through the bits of x, adding 2 k y to the result if the kth. bit of x is 1. 

The value 2 k y can be computed by left-shifting y by k. Since we cannot use add 
directly, we must implement it. We apply the grade-school algorithm for addition to 
the binary case, i.e., compute the sum bit-by-bit, and "rippling" the carry along. 

As an example, we show how to multiply 13 = (1101) 2 and 9 = (1001) 2 using the 
algorithm described above. In the first iteration, since the LSB of 13 is 1, we set the 
result to (1001) 2 . The second bit of (1101) 2 is 0, so we move on to the third bit. This 


51 



bit is 1, so we shift (1001) 2 to the left by 2 to obtain (100100) 2 , which we add to (1001) 2 
to get (101101) 2 . The fourth and final bit of (1101) 2 is 1, so we shift (1001) 2 to the left 
by 3 to obtain (1001000) 2 , which we add to (101101) 2 to get (1110101) 2 = 117. 

Each addition is itself performed bit-by-bit. For example, when adding (101101) 2 
and (1001000) 2 , the LSB of the result is 1 (since exactly one of the two LSBs of the 
operands is 1). The next bit is 0 (since both the next bits of the operands are 0). The 
next bit is 1 (since exactly one of the next bits of the operands is 1). The next bit is 
0 (since both the next bits of the operands are 1). We also "carry" a 1 to the next 
position. The next bit is 1 (since the carry-in is 1 and both the next bits of the operands 
are 0). The remaining bits are assigned similarly. 

public static long multiply (long x, long y) { 
long sum = <9; 
while (x != ®) { 

// Examines each bit of x. 

if C(x & 1) != ®) { 

sum = add(sum, y) ; 

} 

x >>>= 1; 

y <<= l; 

} 

return sum; 


private static long add(long a, long b) { 

long sum = ®, carryin = ®, k = 1, tempA = a, tempB = b; 
while (tempA != ® || tempB != ®) { 
long ak = a & k, bk = b & k; 

long carryout = (ak & bk) | (ak & carryin) | (bk & carryin); 

sum |= (ak A bk A carryin); 

carryin = carryout « 1; 

k «= 1; 

tempA >»= 1; 

tempB >»= 1; 

} 

return sum | carryin; 


The time complexity of addition is 0(ri), where n is the width of the operands. Since 
we do n additions to perform a single multiplication, the total time complexity is 
0(n 2 ). 

5.6 Compute x/y 

Given two positive integers, compute their quotient, using only the addition, sub¬ 
traction, and shifting operators. 

Hint: Relate x/y to (x - y)/y. 

Solution: A brute-force approach is to iteratively subtract y from x until what remains 
is less than y. The number of such subtractions is exactly the quotient, x/y, and the 
remainder is the term that's less than y. The complexity of the brute-force approach 
is very high, e.g., when y - 1 and x = 2 31 - 1, it will take 2 31 - 1 iterations. 
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A better approach is to try and get more work done in each iteration. For example, 
we can compute the largest k such that 2 k y < x, subtract 2 k y from x, and add 2 k to the 
quotient. For example, if x = (1011)2 and y = (10)2, then k- 2, since 2 X 2 2 < 11 and 
2 X 2 3 > 11. We subtract (1000) 2 from (1011) 2 to get (11)2, add 2 k = 2 2 = (100) 2 to the 
quotient, and continue by updating x to (11)2- 

The advantage of using 2 k y is that it can be computed very efficiently using shifting, 
and x is at least halved in each iteration. If it takes n bits to represent x/y, there are 
0(n) iterations. If the largest k such that 2 k y < x is computed by iterating through k, 
each iteration has time complexity 0(n). This leads to an 0(n 2 ) algorithm. 

A better way to find the largest k in each iteration is to recognize that it keeps 
decreasing. Therefore, instead of testing in each iteration whether 2°y,2 1 y,2 2 y,... 
is less than or equal to x, after we initially find the largest k such that 2 k y < x, in 
subsequent iterations we test 2* _1 y, 2^~ 2 y, 2 k ~ 3 y, ... with x. 

For the example given earlier, after setting the quotient to (100)2 we continue with 
(11) 2 . Now the largest k such that 2 k y < (11)2 is 0, so we add 2° = (1) 2 to the quotient, 
which is now (101) 2 . We continue with (11)2 - (10)2 = (1)2- Since (1) 2 < y, we are 
done—the quotient is (101)2 and the remainder is (1)2. 

public static long divide(long x, long y) { 
long result = ®; 
int power = 32; 
long yPower = y « power; 
while (x >= y) { 

while (yPower > x) { 
yPower »>= 1; 

--power; 

} 

result += 1L « power; 
x -= yPower; 

} 

return result; 

} 


In essence, the program applies the grade-school division algorithm to binary num¬ 
bers. With each iteration, we process an additional bit. Therefore, assuming individ¬ 
ual shift and add operations take 0(1) time, the time complexity is 0(n). 


5.7 Compute x y 

Write a program that takes a double x and an integer y and returns x y . You can ignore 
overflow and underflow. 

Hint: Exploit mathematical properties of exponentiation. 

Solution: First, assume y is nonnegative. The brute-force algorithm is to form 
x 2 = x X x, then x 3 = x 2 X x, and so on. This approach takes y — 1 multiplications, 
which is 0(2 n ), where n is number of bits in the integer type. 

The key to efficiency is to try and get more work done with each multiplication, 
thereby using fewer multiplications to accomplish the same result. For example, to 
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compute l.l 21 , instead of starting with 1.1 and multiplying by 1.1 20 times, we could 
multiply 1.1 by l.l 2 = 1.21 10 times for a total of 11 multiplications (one to compute 

1.1 2 , and 10 additional multiplications by 1.21). We can do still better by computing 

1.1 3 , l.l 4 , etc. 

When y is a power of 2, the approach that uses fewest multiplications is iterated 

squaring, i.e., forming x, x 2 , (x 2 ) 2 = x 4 , (x 4 ) 2 = x 8 , _ To develop an algorithm that 

works for general y, it is instructive to look at the binary representation of y, as well 
as properties of exponentiation, specifically x yo+yi = x VQ • x Vl . 

We begin with some small concrete instances, first assuming that y is nonnegative. 
For example, x (1010)2 = *(ioi) 2 +(ioi )2 - * (101)2 x x (101)2 . Similarly, x (101)2 = x (100)2+(1)2 = 
x (100)2 x x = x (10)2 x x (10)2 x x. 

Generalizing, if the least significant bit of y is 0, the result is (x* v/2 ) 2 ; otherwise, it is 
xX (xV /2 ) 2 . This gives us a recursive algorithm for computing xy when y is nonnegative. 

The only change when y is negative is replacing x by 1/x and y by -y. In the im¬ 
plementation below we replace the recursion with a while loop to avoid the overhead 
of function calls. 

public static double power (double x, int y) { 
double result = 1.®; 
long power = y; 
if (y < ®) ( 
power = -power; 
x = 1.8 / x; 

} 

while (power != ®) { 

if ((power & 1) != ®) { 
result *= x; 

} 

x *= x ; 
power >»= 1; 

} 

return result; 

} 


The number of multiplications is at most twice the index of y's MSB, implying an 
0(n) time complexity. 

5.8 Reverse digits 

Write a program which takes an integer and returns the integer corresponding to the 
digits of the input written in reverse order. For example, the reverse of 42 is 24, and 
the reverse of -314 is -413. 

Hint: How would you solve the same problem if the input is presented as a string? 

Solution: The brute-force approach is to convert the input to a string, and then 
compute the reverse from the string by traversing it from back to front. For example, 
( 1100)2 is the decimal number 12, and the answer for ( 1100)2 can be computed by 
traversing the string "12" in reverse order. 

Closer analysis shows that we can avoid having to form a string. Consider the 
input 1132. The first digit of the result is 2, which we can obtain by taking the input 
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modulo 10. The remaining digits of the result are the reverse of 1132/10 = 113. 
Generalizing, let the input be k. If k > 0, then k mod 10 is the most significant digit 
of the result and the subsequent digits are the reverse of Continuing with the 
example, we iteratively update the result and the input as 2 and 113, then 23 and 11, 
then 231 and 1, then 2311. 

For general k, we record its sign, solve the problem for \k\, and apply the sign to 
the result. 

public static long reverse (int x) { 
long result = ®; 
long xRemaining = Math.abs(x); 
while (xRemaining != ®) { 

result = result * 1® + xRemaining % 1®; 
xRemaining /= 1®; 

} 

return x < ® ? -result : result; 


The time complexity is 0(n), where n is the number of digits in k. 


5.9 Check if a decimal integer is a palindrome 

A palindromic string is one which reads the same forwards and backwards, e.g., 
"redivider". In this problem, you are to write a program which determines if the 
decimal representation of an integer is a palindromic string. For example, your 
program should return true for the inputs 0,1,7,11,121,333, and 2147447412, and 
false for the inputs -1,12,100, and 2147483647. 

Write a program that takes an integer and determines if that integer's representation 
as a decimal string is a palindrome. 

Hint: It's easy to come up with a simple expression that extracts the least significant digit. Can 
you find a simple expression for the most significant digit? 

Solution: First note that if the input is negative, then its representation as a decimal 
string cannot be palindromic, since it begins with a -. 

A brute-force approach would be to convert the input to a string and then iterate 
through the string, pairwise comparing digits starting from the least significant digit 
and the most significant digit, and working inwards, stopping if there is a mismatch. 
The time and space complexity are 0(n), where n is the number of digits in the input. 

We can avoid the 0(n) space complexity used by the string representation by 
directly extracting the digits from the input. The number of digits, n, in the input's 
string representation is the log (base 10) of the input value, x. To be precise, n = 
|_log 10 xj +1. Therefore, the least significant digit is x mod 10, and the most significant 
digit is x/10" -1 . In the program below, we iteratively compare the most and least 
significant digits, and then remove them from the input. For example, if the input is 
151751, we would compare the leading and trailing digits, 1 and 1. Since these are 
equal, we update the value to 5175. The leading and trailing digits are equal, so we 
update to 17. Now the leading and trailing are unequal, so we return false. If instead 
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the number was 157751, the final compare would be of 7 with 7, so we would return 
true. 

public static boolean isPalindromeNumber (int x) { 
if (x < ®) { 
return false; 

} 

final int numDigits = (int)(Math.floor(Math.logl®(x))) + 1; 
int msdMask = (int)Math.pow(l®, numDigits - 1); 
for (int i = ®; i < (numDigits / 2); ++i) { 
if (x / msdMask != x % 1®) { 

return false; 

} 

x %= msdMask; // Remove the most significant digit of x. 
x /= 1®; // Remove the least significant digit of x. 
msdMask /= 1®®; 

} 

return true; 


The time complexity is 0(n), and the space complexity is 0(1). Alternatively, we 
could use Solution 5.8 on Page 54 to reverse the digits in the number and see if it is 
unchanged. 


5.10 Generate uniform random numbers 

This problem is motivated by the following scenario. Six friends have to select 
a designated driver using a single unbiased coin. The process should be fair to 
everyone. 

How would you implement a random number generator that generates a random 
integer i between a and b, inclusive, given a random number generator that produces 
zero or one with equal probability? All values in [a, b] should be equally likely. 

Hint: How would you mimic a three-sided coin with a two-sided coin? 

Solution: Note that it is easy to produce a random integer between 0 and 2' - 1, 
inclusive: concatenate i bits produced by the random number generator. For example, 
two calls to the random number generator will produce one of (00) 2 , (01) 2 , (10) 2 , ( 11 ) 2 - 
These four possible outcomes encode the four integers 0,1,2,3, and all of them are 
equally likely. 

For the general case, first note that it is equivalent to produce a random integer 
between 0 and b - a, inclusive, since we can simply add a to the result. If b - a is equal 
to 2 l — 1, for some i, then we can use the approach in the previous paragraph. 

If b - a is not of the form 2 l - 1, we find the smallest number of the form 2 1 - 1 that 
is greater than b-a. We generate an i -bit number as before. This i -bit number may or 
may not lie between 0 and b - a, inclusive. If it is within the range, we return it—all 
such numbers are equally likely. If it is not within the range, we try again with i new 
random bits. We keep trying until we get a number within the range. 
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For example, to generate a random number corresponding to a dice roll, 
i.e., a number between 1 and 6, we begin by making three calls to the ran¬ 
dom number generator (since 2 2 - 1 < (6 - 1) < 2 3 - 1). If this yields one of 
( 000 ) 2 ,( 001 ) 2 ,( 010 ) 2 ,( 011 ) 2 ,( 100 ) 2 ,( 101 ) 2 , we return 1 plus the corresponding value. 
Observe that all six values between 1 and 6, inclusive, are equally likely to be re¬ 
turned. If the three calls yields one of (110)2,(111)2, we make three more calls. Note 
that the probability of having to try again is 2/8, which is less than half. Since succes¬ 
sive calls are independent, the probability that we require many attempts diminishes 
very rapidly, e.g., the probability of not getting a result in 10 attempts is (2/8) 10 which 
is less than one-in-a-million. 


public static int uniformRandom(int lowerBound, int upperBound) { 
int numberOfOutcomes = upperBound - lowerBound + 1, result; 
do { 

result = Q; 

for (int i = ®; (1 « i) < numberOfOutcomes; ++i) { 

// zeroOneRandom() is the provided random number generator. 
result = (result « 1) | zeroOneRandom(); 

} 

} while (result >= numberOfOutcomes); 
return result + lowerBound; 


To analyze the time complexity, let t = b - a + 1. The probability that we succeed in 
the first try is f/2'. Since 2* is the smallest power of 2 greater than or equal to t, it 
must be less than 2t. (An easy way to see this is to consider the binary representation 
of t and 2 1.) This implies that f/2' > t/2t = (1/2). Hence the probability that we do 
not succeed on the first try is 1 - t/ 2' < 1/2. Since successive tries are independent, 
the probability that more than k tries are needed is less than or equal to l/2 k . Hence, 
the expected number of tries is not more than 1 + 2(1/2) 1 + 3(1/2) 2 + .... The series 
converges, so the number of tries is 0(1). Each try makes fig (b - a + 1)] calls to the 
0 /1-valued random number generator. Assuming the 0/1-valued random number 
generator takes 0(1) time, the time complexity is 0(\g(b - a + 1)). 


5.11 Rectangle intersection 

This problem is concerned with rectangles whose sides are parallel to the X-axis and 
Y-axis. See Figure 5.2 for examples. 



Figure 5.2: Examples of XY-aligned rectangles. 
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Write a program which tests if two rectangles have a nonempty intersection. If the 
intersection is nonempty, return the rectangle formed by their intersection. 

Hint: Think of the X and Y dimensions independently. 

Solution: Since the problem leaves it unspecified, we will treat the boundary as part 
of the rectangle. This implies, for example, rectangles A and B in Figure 5.2 on the 
preceding page intersect. 

There are many qualitatively different ways in which rectangles can intersect, e.g., 
they have partial overlap (D and F), one contains the other (F and G), they share a 
common side (D and £), they share a common corner (A and B), they form a cross (B 
and C), they form a tee (F and H), etc. The case analysis is quite tricky. 

A better approach is to focus on conditions under which it can be guaranteed that 
the rectangles do not intersect. For example, the rectangle with left-most lower point 
(1,2), width 3, and height 4 cannot possibly intersect with the rectangle with left-most 
lower point (5,3), width 2, and height 4, since the X-values of the first rectangle range 
from 1 to 1 + 3 = 4, inclusive, and the X-values of the second rectangle range from 5 
to 5 + 2 = 7, inclusive. 

Similarly, if the Y-values of the first rectangle do not intersect with the Y-values of 
the second rectangle, the two rectangles cannot intersect. 

Equivalently, if the set of X-values for the rectangles intersect and the set of Y- 
values for the rectangles intersect, then all points with those X- and Y-values are 
common to the two rectangles, so there is a nonempty intersection. 

public static class Rectangle { 
int x, y, width, height; 

public Rectangle (int x, int y, int width, int height) { 
this.x = x; 
this.y = y; 
this. width = width; 
this. height = height; 

} 


public static Rectangle intersectRectangle(Rectangle R1, Rectangle R2) { 
if (!islntersect(R1, R2)) { 

return new Rectangle(Q, ®, -1, -1); // No intersection. 

} 

return new Rectangle( 

Math.max(Rl.x, R2.x), Math.max(R1.y, R2.y), 

Math.min(Rl.x + Rl.width, R2.x + R2.width) - Math.max(R1.x, R2.x), 
Math.min(Rl.y + Rl.height, R2.y + R2.height) - Math.max(Rl.y, R2.y)); 


public static boolean islntersect(Rectangle Rl, Rectangle R2) { 
return Rl.x <= R2.x + R2.width && Rl.x + Rl.width >= R2.x 

&& Rl.y <= R2.y + R2.height &<& Rl.y + Rl.height >= R2.y; 

} 


The time complexity is 0(1), since the number of operations is constant. 
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Variant: Given four points in the plane, how would you check if they are the vertices 
of a rectangle? 

Variant: How would you check if two rectangles, not necessarily aligned with the X 
and Y axes, intersect? 
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Chapter 


The machine can alter the scanned symbol and its behavior 
is in part determined by that symbol, but the symbols on the 
tape elsewhere do not affect the behavior of the machine. 

— "Intelligent Machinery," 
A. M. Turing, 1948 


The simplest data structure is the array, which is a contiguous block of memory. It is 
usually used to represent sequences. Given an array A, A[i] denotes the (i + l)th object 
stored in the array. Retrieving and updating A[i] takes 0(1) time. Insertion into a full 
array can be handled by resizing, i.e., allocating a new array with additional memory 
and copying over the entries from the original array. This increases the worst-case 
time of insertion, but if the new array has, for example, a constant factor larger than the 
original array, the average time for insertion is constant since resizing is infrequent. 
Deleting an element from an array entails moving all successive elements one over 
to the left to fill the vacated space. For example, if the array is (2,3,5,7,9,11,13,17), 
then deleting the element at index 4 results in the array (2,3,5,7,11,13,17,0). (We do 
not care about the last value.) The time complexity to delete the element at index i 
from an array of length n is 0(n - i). 

Array boot camp 

The following problem gives good insight into working with arrays: Your input is an 
array of integers, and you have to reorder its entries so that the even entries appear 
first. This is easy if you use 0(n) space, where n is the length of the array. However, 
you are required to solve it without allocating additional storage. 

When working with arrays you should take advantage of the fact that you can 
operate efficiently on both ends. For this problem, we can partition the array into 
three subarrays: Even, Unclassified, and Odd, appearing in that order. Initially 
Even and Odd are empty, and Unclassified is the entire array. We iterate through 
Unclassified, moving its elements to the boundaries of the Even and Odd subarrays 
via swaps, thereby expanding Even and Odd, and shrinking Unclassified. 

public static void evenOdd(int[] A) { 

int nextEven = Q, nextOdd = A.length - 1; 
while (nextEven < nextOdd) { 
if (A[nextEven] % 2 == Q) { 
nextEven++; 

} else { 

int temp = A[nextEven]; 

A[nextEven] = A[next0dd]; 
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A[nextOdd--] = temp; 


} 

} 


} 


The additional space complexity is clearly 0(1 )—a couple of variables that hold 
indices, and a temporary variable for performing the swap. We do a constant amount 
of processing per entry, so the time complexity is 0(n). 


Array problems often have simple brute-force solutions that use 0(n) space, but 
subtler solutions that use the array itself to reduce space complexity to 0(1). 
[Problem 6.1] 

Filling an array from the front is slow, so see if it's possible to write values from 
the back. [Problem 6.2] 

Instead of deleting an entry (which requires moving all entries to its right), con¬ 
sider overwriting it. [Problem 6.5] 

When dealing with integers encoded by an array consider processing the digits 
from the back of the array. Alternately, reverse the array so the least-significant 
digit is the first entry. [Problem 6.3] 

Be comfortable with writing code that operates on subarrays. [Problem 6.10] 

It's incredibly easy to make off-by-1 errors when operating on arrays. [Prob¬ 
lems 6.4 and 6.17] 

Don't worry about preserving the integrity of the array (sortedness, keeping equal 
entries together, etc.) until it is time to return. [Problem 6.5] 

An array can serve as a good data structure when you know the distribution of 
the elements in advance. For example, a Boolean array of length W is a good 
choice for representing a subset of {0,1,..., W - 1}. (When using a Boolean array 
to represent a subset of {1,2,3,... ,n}, allocate an array of size n + 1 to simplify 
indexing.) [Problem 6.8]. 

When operating on 2D arrays, use parallel logic for rows and for columns. [Prob¬ 
lem 6.17] 

Sometimes it's easier to simulate the specification, than to analytically solve for 
the result. For example, rather than writing a formula for the z-th entry in the 
spiral order for an n X n matrix, just compute the output from the beginning. 
[Problems 6.17 and 6.19] 


Know your array libraries 

The basic array type in Java is fixed-size. You should know the Java Arrays utilities 
very intimately; these simplify working with the basic array type. The ArrayList 
type implements a dynamically resized array, and is a good replacement for the basic 
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Java array; it's more flexible, and has more API methods. Be sure to read the review 
of Li st, ArrayLi st, and Collecti ons on Page 114. 

• Know the syntax for allocating and initializating an array, i.e., new 
int [] {1,2,3}. 

• Understand how to instantiate a 2D array—new Integer [3] [] creates an array 
which will hold three rows, each of these must be explicitly assigned. 

• Don't forget the length of an array is given by the length field, unlike 
Collections, which uses the size() method, and String, which use the 
length() method. 

• The Arrays class consists of a set of static utility methods. All of them are 
important: asListO (more on this later), binarySearch(A, 641), copyOf(A), 
copyOfRange(A, 1,5), equals(A, B), fill(A, 42), find(A, 28), sort(A), 
sort(A,cmp), toString(). 

- Understand the variants of these methods, e.g., how to create a copy of a 
subarray. 

- Understand what "deep" means when checking equality of arrays, and 
hashing them. 

Both Arrays and Collections have binarySearchO and sort() methods. These are 
nuanced—we discuss them in detail on Page 189 and on Page 235. 

6.1 The Dutch national flag problem 

The quicksort algorithm for sorting arrays proceeds recursively—it selects an element 
(the "pivot"), reorders the array to make all the elements less than or equal to the pivot 
appear first, followed by all the elements greater than the pivot. The two subarrays 
are then sorted recursively. 

Implemented naively, quicksort has large run times and deep function call stacks 
on arrays with many duplicates because the subarrays may differ greatly in size. One 
solution is to reorder the array so that all elements less than the pivot appear first, 
followed by elements equal to the pivot, followed by elements greater than the pivot. 
This is known as Dutch national flag partitioning, because the Dutch national flag 
consists of three horizontal bands, each in a different color. 

As an example, assuming that black precedes white and white precedes gray. 
Figure 6.1(b) on the facing page is a valid partitioning for Figure 6.1(a) on the next 
page. If gray precedes black and black precedes white. Figure 6.1(c) on the facing 
page is a valid partitioning for Figure 6.1(a) on the next page. 

Generalizing, suppose A = (0,1,2,0,2,1,1), and the pivot index is 3. ThenA[3] = 0, 
so (0,0,1,2,2,1,1) is a valid partitioning. For the same array, if the pivot index is 2, 
then A[ 2] = 2, so the arrays (0,1,0,1,1,2,2) as well as langle 0,0,1,1,1,2,2) are valid 
partitionings. 

Write a program that takes an array A and an index i into A, and rearranges the 
elements such that all elements less than A[i] (the "pivot") appear first, followed by 
elements equal to the pivot, followed by elements greater than the pivot. 

Hint: Think about the partition step in quicksort. 
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(a) Before partitioning. (b) A three-way partitioning resem- (c) Another three-way partitioning: 

bling the Dutch national flag. the Russian national flag. 

Figure 6.1: Illustrating the Dutch national flag problem. 


Solution: The problem is trivial to solve with 0(n) additional space, where n is the 
length of A. We form three lists, namely, elements less than the pivot, elements equal 
to the pivot, and elements greater than the pivot. Consequently, we write these values 
into A. The time complexity is 0{n). 

We can avoid using 0(n) additional space at the cost of increased time complexity 
as follows. In the first stage, we iterate through A starting from index 0, then index 1, 
etc. In each iteration, we seek an element smaller than the pivot—as soon as we find 
it, we move it to the subarray of smaller elements via an exchange. This moves all 
the elements less than the pivot to the start of the array. The second stage is similar 
to the first one, the difference being that we move elements greater than the pivot to 
the end of the array. Code illustrating this approach is shown below. 

public static enum Color { RED, WHITE, BLUE } 

public static void dutchFlagPartition(int pivotlndex, List<Color> A) { 

Color pivot = A.get(pivotlndex); 

// First pass: group elements smaller than pivot. 
for (int i = Q; i < A.sizeO; ++i) { 

// Look for a smaller element. 

for (int j = i + 1; j < A.sizeO; ++j) { 

if (A.get(j).ordinal() < pivot.ordinal()) { 

Collections.swap(A, i, j); 
break; 

} 

} 

} 

// Second pass: group elements larger than pivot. 

for (int i = A.sizeO - 1; i >= ® && A . get (i) . ordinal () >= pivot. ordinal () ; 

-i) ( 

// Look for a larger element. Stop when we reach an element less 
// than pivot, since first pass has moved them to the start of A. 
for (int j=i-l;j>=®&& A.get(j).ordinal() >= pivot.ordinal(); 

--j) { 

if (A.get(j).ordinal() > pivot.ordinal()) { 

Collections.swap(A, i, j); 
break; 

} 

} 

} 

} 
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The additional space complexity is now 0(1), but the time complexity is 0(n 2 ), 
e.g., if i = n/2 and all elements before i are greater than A[i\, and all elements after i 
are less than A[i\. Intuitively, this approach has bad time complexity because in the 
first pass when searching for each additional element smaller than the pivot we start 
from the beginning. However, there is no reason to start from so far back—we can 
begin from the last location we advanced to. (Similar comments hold for the second 
pass.) 

To improve time complexity, we make a single pass and move all the elements 
less than the pivot to the beginning. In the second pass we move the larger elements 
to the end. It is easy to perform each pass in a single iteration, moving out-of-place 
elements as soon as they are discovered. 

public static enum Color { RED, WHITE, BLUE } 

public static void dutchFlagPartition(int pivotlndex, List<Color> A) { 

Color pivot = A.get(pivotlndex); 

// First pass: group elements smaller than pivot. 
int smaller = ®; 

for (int i = ®; i < A.size(); ++i) { 

if (A.get(i).ordinal() < pivot.ordinal()) { 

Collections.swap(A, smaller++, i); 

} 

} 

// Second pass: group elements larger than pivot. 
int larger = A.size() - 1; 

for (int i = A.sizeO - 1; i >= ® && A. get (i) . ordinal () >= pivot. ordinal () ; 

-i) ( 

if (A.get(i).ordinal() > pivot.ordinal()) { 

Collections.swap(A, larger--, i); 

> 

} 

} 


The time complexity is 0(n) and the space complexity is 0(1). 

The algorithm we now present is similar to the one sketched above. The main 
difference is that it performs classification into elements less than, equal to, and 
greater than the pivot in a single pass. This reduces runtime, at the cost of a trickier 
implementation. We do this by maintaining four subarrays: bottom (elements less 
than pivot), middle (elements equal to pivot), unclassified, and top (elements greater 
than pivot). Initially, all elements are in unclassified. We iterate through elements in 
unclassified, and move elements into one of bottom, middle, and top groups according 
to the relative order between the incoming unclassified element and the pivot. 

As a concrete example, suppose the array is currently A = (-3,0, -1,1,1, ?, ?, ?, 4,2), 
where the pivot is 1 and ? denotes unclassified elements. There are three possibilities 
for the first unclassified element, A[5]. 

• A[5] is less than the pivot, e.g., A[5] = -5. We exchange it with the first 1, i.e., 
the new array is (-3,0, -1, -5,1,1, ?, ?, 4,2). 

• A[5] is equal to the pivot, i.e., A[5] = 1. We do not need to move it, we just ad¬ 
vance to the next unclassified element, i.e., the array is (-3,0, -1,1,1,1, ?, ?, 4,2). 

• A[5] is greater than the pivot, e.g., A[5] = 3. We exchange it with the last 
unclassified element, i.e., the new array is (-3,0, -1,1,1, ?, ?, 3,4,2). 
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Note how the number of unclassified elements reduces by one in each case, 
public static enum Color { RED, WHITE, BLUE } 

public static void dutchFlagPartition(int pivotlndex, List<Color> A) { 
Color pivot = A.get(pivotlndex); 

/* * 

* Keep the following invariants during partitioning: 

* bottom group: A.sublist(Q, smaller). 

* middle group: A.sublist(smaller, equal). 

* unclassified group: A.subList(equal, larger). 

* top group: A . subList (larger, A.sizeO). 

V 

int smaller = Q, equal = Q, larger = A.sizeO; 

// Keep iterating as long as there is an unclassified element. 
while (equal < larger) { 

// A.get(equal) is the incoming unclassified element. 
if (A.get(equal).ordinal() < pivot.ordinal()) { 

Collections.swap(A, smaller++, equal++); 

} else if (A.get(equal).ordinal() == pivot.ordinal()) { 

++equal; 

} else { // A.get(equal) > pivot. 

Collections.swap(A, equal, --larger); 

} 

} 

} 


Each iteration decreases the size of unclassified by 1, and the time spent within each 
iteration is <9(1), implying the time complexity is 0(n). The space complexity is clearly 

o(D- 

Variant: Assuming that keys take one of three values, reorder the array so that all 
objects with the same key appear together. The order of the subarrays is not important. 
For example, both Figures 6.1(b) and 6.1(c) on Page 63 are valid answers for 
Figure 6.1(a) on Page 63. Use 0(1) additional space and 0(n) time. 

Variant: Given an array A of n objects with keys that takes one of four values, reorder 
the array so that all objects that have the same key appear together. Use <9(1) additional 
space and 0(n) time. 

Variant: Given an array A of n objects with Boolean-valued keys, reorder the array so 
that objects that have the key false appear first. Use 0(1) additional space and 0(n) 
time. 

Variant: Given an array A of n objects with Boolean-valued keys, reorder the array so 
that objects that have the key false appear first. The relative ordering of objects with 
key true should not change. Use <9(1) additional space and 0(n) time. 


6.2 Increment an arbitrary-precision integer 

Write a program which takes as input an array of digits encoding a decimal number 
D and updates the array to represent the number D + 1. For example, if the input 
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is (1,2,9) then you should update the array to (1,3,0). Your algorithm should work 
even if it is implemented in a language that has finite-precision arithmetic. 

Hint: Experiment with concrete examples. 

Solution: A brute-force approach might be to convert the array of digits to the 
equivalent integer, increment that, and then convert the resulting value back to an 
array of digits. For example, if the array is (1,2,9), we would derive the integer 129, 
add one to get 130, then extract its digits to form (1,3,0). When implemented in a 
language that imposes a limit on the range of values an integer type can take, this 
approach will fail on inputs that encode integers outside of that range. 

We can avoid overflow issues by operating directly on the array of digits. Specifi¬ 
cally, we mimic the grade-school algorithm for adding numbers, which entails adding 
digits starting from the least significant digit, and propagate carries. If the result has 
an additional digit, e.g., 99 + 1 = 100, all digits have to be moved to the right by one. 

For the given example, we would update 9 to 0 with a carry-out of 1. We update 2 
to 3 (because of the carry-in). There is no carry-out, so we stop—the result is (1,3,0). 

public static List<Integer> plusOne(Listdnteger> A) { 
int n = A.sizeO - 1; 

A.set(n, A.get(n) + 1); 

for (int i = n; i > ® && A.get(i) == 1©; --i) { 

A.set(i , Q) ; 

A.set(i - 1, A.get(i - 1) + 1); 

} 

if (A.get(®) == 1®) { 

// Need additional digit as the most significant digit (i.e. f A.get(9)) 

// has a carry-out. 

A.set(®, Q) ; 

A.add(®, 1); 

} 

return A; 

} 


The time complexity is 0{ri), where n is the length of A. 

Variant: Write a program which takes as input two strings s and t of bits encoding 
binary numbers B s and B t , respectively, and returns a new string of bits representing 
the number B s + B t . 


6.3 Multiply two arbitrary-precision integers 

Certain applications require arbitrary precision arithmetic. One way to achieve 
this is to use arrays to represent integers, e.g., with one digit per array entry, 
with the most significant digit appearing first, and a negative leading digit denot¬ 
ing a negative integer. For example, (1,9,3,7,0,7,7,2,1) represents 193707721 and 
(-7,6,1,8,3,8,2,5,7,2,8,7) represents -761838257287. 

Write a program that takes two arrays representing integers, and re¬ 
turns an integer representing their product. For example, since 

193707721 x -761838257287 = -147573952589676412927, if the inputs are 
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(1,9,3,7,0,7,7,2,1) and (-7,6,1,8,3,8,2,5,7,2,8,7), your function should return 
(-1,4,7,5,7,3,9,5,2,5,8,9,6,7,6,4,1,2,9,2,7). 

Hint: Use arrays to simulate the grade-school multiplication algorithm. 

Solution: As in Solution 6.2 on Page 65, the possibility of overflow precludes us from 
converting to the integer type. 

Instead we can use the grade-school algorithm for multiplication which consists 
of multiplying the first number by each digit of the second, and then adding all the 
resulting terms. 

From a space perspective, it is better to incrementally add the terms rather than 
compute all of them individually and then add them up. The number of digits 
required for the product is at most n + m for n and m digit operands, so we use an 
array of size n + m for the result. 

For example, when multiplying 123 with 987, we would form 7 X 123 = 861, then 
we would form 8 X 123 X 10 = 9840, which we would add to 861 to get 10701. Then 
we would form 9 X 123 X 100 = 110700, which we would add to 10701 to get the final 
result 121401. (All numbers shown are represented using arrays of digits.) 

public static List<Integer> multiply(List<Integer> numl, Listdnteger> num2) { 
final int sign = numl.get(Q) < © A num2.get(®) < ® ? -1 : 1; 
numl.set (®, Math.abs(numl.get (®))) ; 
num2 . set (® , Math . abs (num2 . get (®) ) ) ; 

List<Integer> result 

= new ArrayList<>(Collections.nCopies(numl. size () + num2.size(), ®)); 
for (int i = numl.size() - 1; i >= ®; --i) { 
for (int j = num2.size() - 1; j >= ®; --j) { 
result.set(i + j + 1, 

result.get(i + j + 1) + numl.get(i) * num2.get(j)); 
result.set(i + j, result.get(i + j) + result.get(i + j + 1) / 1®); 
result.set(i + j + 1, result.get(i + j + 1) % 1®); 

} 

} 

// .Remove the leading zeroes. 
int first_not_zero = ®; 

while (first_not_zero < result.size() && result.get(first_not_zero) == ®) { 

++first_not_zero; 

} 

result = result.subList(first_not_zero, result.size()); 
if (result.isEmpty()) { 
return Arrays.asList(®) ; 

} 

result.set(®, result.get(®) * sign); 
return result; 


There are m partial products, each with at most n + 1 digits. We perform (9(1) 
operations on each digit in each partial product, so the time complexity is 0(nm). 
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6.4 Advancing through an array 


In a particular board game, a player has to try to advance through a sequence of 
positions. Each position has a nonnegative integer associated with it, representing 
the maximum you can advance from that position in one move. You begin at the first 
position, and win by getting to the last position. For example, let A = (3,3,1,0,2,0,1) 
represent the board game, i.e., the ith entry in A is the maximum we can advance 
from i. Then the game can be won by the following sequence of advances through 
A: take 1 step from A[0] to A[ 1], then 3 steps from A[ 1] to A[ 4], then 2 steps from 
A[ 4] to A[ 6], which is the last position. Note that A[0] = 3 > 1, A[ 1] = 3 > 3, and 
A [4] = 2 > 2, so all moves are valid. If A instead was (3,2,0,0,2,0,1), it would not 
possible to advance past position 3, so the game cannot be won. 

Write a program which takes an array of n integers, where A[i] denotes the maximum 
you can advance from index i, and returns whether it is possible to advance to the 
last index starting from the beginning of the array. 

Hint: Analyze each location, starting from the beginning. 

Solution: It is natural to try advancing as far as possible in each step. This approach 
does not always work, because it potentially skips indices containing large entries. 
For example, if A = (2,4,1,1,0,2,3), then it advances to index 2, which contains a 
1, which leads to index 3, after which it cannot progress. However, advancing to 
index 1, which contains a 4 lets us proceed to index 5, from which we can advance to 
index 6. 

The above example suggests iterating through all entries in A. As we iterate 
through the array, we track the furthest index we know we can advance to. The 
furthest we can advance from index i is i + A[i\. If, for some i before the end of the 
array, i is the furthest index that we have demonstrated that we can advance to, we 
cannot reach the last index. Otherwise, we reach the end. 

For example, if A = (3,3,1,0,2,0,1), we iteratively compute the furthest we can 
advance to as 0,3,4,4,4,6,6,7, which reaches the last index, 6. If A = (3,2,0,0,2,0,1), 
we iteratively update the furthest we can advance to as 0,3,3,3,3, after which we 
cannot advance, so it is not possible to reach the last index. 

The code below implements this algorithm. Note that it is robust with respect to 
negative entries, since we track the maximum of how far we proved we can advance 
to and i + A[i]. 

public static boolean canReachEnd(List<Integer> maxAdvanceSteps) { 
int furthestReachSoFar = Q, lastlndex = maxAdvanceSteps.size() - 1; 
for (int i = Q; i <= furthestReachSoFar && furthestReachSoFar < lastlndex; 

++i) 1 

furthestReachSoFar 

= Math.max(furthestReachSoFar, i + maxAdvanceSteps.get(i)); 

} 

return furthestReachSoFar >= lastlndex; 

} 


The time complexity is 0(n), and the additional space complexity (beyond what is 
used for A) is three integer variables, i.e., 0(1). 
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Variant: Write a program to compute the minimum number of steps needed to ad¬ 
vance to the last location. 


6.5 Delete duplicates from a sorted array 

This problem is concerned with deleting repeated elements from a sorted array. 
For example, for the array (2,3,5,5,7,11,11,11,13), then after deletion, the array is 
(2,3,5,7,11,13,0,0,0). After deleting repeated elements, there are 6 valid entries. 
There are no requirements as to the values stored beyond the last valid element. 

Write a program which takes as input a sorted array and updates it so that all dupli¬ 
cates have been removed and the remaining elements have been shifted left to fill the 
emptied indices. Return the number of valid elements. Many languages have library 
functions for performing this operation—you cannot use these functions. 

Hint: There is an 0(n) time and 0(1) space solution. 

Solution: Let A be the array and n its length. If we allow ourselves 0(n) additional 
space, we can solve the problem by iterating through A and recording values that 
have not appeared previously into a hash table. (The hash table is used to determine 
if a value is new.) New values are also written to a list. The list is then copied back 
into A. 

Here is a brute-force algorithm that uses <9(1) additional space—iterate through A, 
testing if A[i] equals A[i + 1], and, if so, shift all elements at and after i + 2 to the left 
by one. When all entries are equal, the number of shifts is (n - 1) + (n - 2) + • • • + 2 + 1, 
i.e., G(n 2 ), where n is the length of the array. 

The intuition behind achieving a better time complexity is to reduce the amount of 
shifting. Since the array is sorted, repeated elements must appear one-after-another, 
so we do not need an auxiliary data structure to check if an element has appeared 
already. We move just one element, rather than an entire subarray, and ensure that 
we move it just once. 

For the given example, (2,3,5,5,7,11,11,11,13), when processing the A[ 3], since 
we already have a 5 (which we know by comparing A[ 3] with A[ 2]), we advance to 
A[ 4]. Since this is a new value, we move it to the first vacant entry, namely A[3]. Now 
the array is (2,3,5,7,7,11,11,11,13), and the first vacant entry is A [4]. We continue 
from A [5]. 

// Returns the number of valid entries after deletion. 
public static int deleteDuplicates(List<Integer> A) { 
if (A.isEmpty()) { 
return Q; 

} 

int writelndex = 1; 

for (int i = 1; i < A.sizeO; ++i) { 

if (!A.get(writelndex - 1).equals(A.get(i))) { 

A.set(writelndex++, A.get(i)); 

} 

} 

return writelndex; 
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} 


The time complexity is 0(n), and the space complexity is 0(1), since all that is needed 
is the two additional variables. 

Variant: Implement a function which takes as input an array and a key, and updates 
the array so that all occurrences of the input key have been removed and the remaining 
elements have been shifted left to fill the emptied indices. Return the number of 
remaining elements. There are no requirements as to the values stored beyond the 
last valid element. 

Variant: Write a program which takes as input a sorted array A of integers and a 
positive integer m, and updates A so that if x appears m times in A it appears exactly 
min(2 ,m) times in A. The update to A should be performed in one pass, and no 
additional storage may be allocated. 


6.6 Buy and sell a stock once 

This problem is concerned with the problem of optimally buying and selling a stock 
once, as described on Page 2. As an example, consider the following sequence of 
stock prices: (310,315,275,295,260,270,290,230,255,250). The maximum profit that 
can be made with one buy and one sell is 30—buy at 260 and sell at 290. Note that 
260 is not the lowest price, nor 290 the highest price. 

Write a program that takes an array denoting the daily stock price, and returns the 
maximum profit that could be made by buying and then selling one share of that 
stock. 

Hint: Identifying the minimum and maximum is not enough since the minimum may appear 
after the maximum height. Focus on valid differences. 

Solution: We developed several algorithms for this problem in the introduction. 
Specifically, on Page 2 we showed how to compute the maximum profit by computing 
the difference of the current entry with the minimum value seen so far as we iterate 
through the array. 

For example, the array of minimum values seen so far for the given example is 
(310,310,275,275,260,260,260,230,230,230). The maximum profit that can be made 
by selling on each specific day is the difference of the current price and the mini¬ 
mum seen so far, i.e., (0,5,0,20,0,10,30,0,25,20). The maximum profit overall is 30, 
corresponding to buying 260 and selling for 290. 

public static double computeMaxProfit(List<Double> prices) { 
double minPrice = Double.MAX_VALUE, maxProfit = ®.®; 
for (Double price : prices) { 

maxProfit = Math.max(maxProfit, price - minPrice); 
minPrice = Math.min(minPrice, price); 

} 

return maxProfit; 

} 
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The time complexity is 0(n) and the space complexity is 0(1), where n is the length 
of the array. 

Variant: Write a program that takes an array of integers and finds the length of a 
longest subarray all of whose entries are equal. 


6.7 Buy and sell a stock twice 

The max difference problem, introduced on Page 1, formalizes the maximum profit 
that can be made by buying and then selling a single share over a given day range. 

Write a program that computes the maximum profit that can be made by buying and 
selling a share at most twice. The second buy must be made on another date after the 
first sale. 

Hint: What do you need to know about the first i elements when processing the (i + l)th element? 

Solution: The brute-force algorithm which examines all possible combinations of 
buy-sell-buy-sell days has complexity 0(n 4 ). The complexity can be improved to 
0(n 2 ) by applying the 0(n) algorithm to each pair of subarrays formed by splitting A. 

The inefficiency in the above approaches comes from not taking advantage of 
previous computations. Suppose we record the best solution for A [0 : /], j between 1 
and n — 1, inclusive. Now we can do a reverse iteration, computing the best solution 
for a single buy-and-sell for A[j : n - 1], j between 1 and n - 1, inclusive. For each 
day, we combine this result with the result from the forward iteration for the previous 
day—this yields the maximum profit if we buy and sell once before the current day 
and once at or after the current day. 

For example, suppose the input array is (12,11,13,9,12,8,14,13,15). Then the 
most profit that can be made with a single buy and sell by Day i (inclusive) is 
F = (0,0,2,2,3,3,6,6,7). Working backwards, the most profit that can be made with 
a single buy and sell on or after Day i is B = (7,7,7,7,7,7,2,2,0). To combine these 
two, we compute M[i] = F[i - 1] + B[i], where F[-l] is taken to be 0 (since the second 
buy must happen strictly after the first sell). This yields M = (7,7,7,9,9,10,5,8,6), 
i.e., the maximum profit is 10. 

public static double buyAndSellStockTwice(List<Double> prices) { 
double maxTotalProfit = Q.Q; 

List<Double> firstBuySellProfits = new ArrayList<>(); 
double minPriceSoFar = Double.MAX_VALUE; 

// Forward phase. For each day, we record maximum profit if we 
// sell on that day. 

for (int i = ®; i < prices. size () ; ++i) { 

minPriceSoFar = Math.min(minPriceSoFar, prices.get(i)); 

maxTotalProfit = Math.max(maxTotalProfit, prices.get(i) - minPriceSoFar); 
firstBuySellProfits.add(maxTotalProfit); 

} 

// Backward phase. For each day, find the maximum profit if we make 

// the second buy on that day. 

double maxPriceSoFar = Double.MIN.VALUE; 
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for (int i = prices. size () - 1; i > ®; --i) { 

maxPriceSoFar = Math.max(maxPriceSoFar, prices.get(i)); 
maxTotalProfit 

= Math.max(maxTotalProfit, maxPriceSoFar - prices.get(i) 

+ firstBuySellProfits.get(i - 1)); 

} 

return maxTotalProfit; 


The time complexity is 0{ri) r and the additional space complexity is 0{n), which is 
the space used to store the best solutions for the subarrays. 

Variant: Solve the same problem in 0(n) time and (9(1) space. 


6.8 Enumerate all primes to n 

A natural number is called a prime if it is bigger than 1 and has no divisors other than 

1 and itself. 

Write a program that takes an integer argument and returns all the primes between 1 
and that integer. For example, if the input is 18, you should return (2,3,5,7,11,13,17). 

Hint: Exclude the multiples of primes. 

Solution: The natural brute-force algorithm is to iterate over all i from 2 to n, where n 
is the input to the program. For each i, we test if i is prime; if so we add it to the result. 
We can use "trial-division" to test if i is prime, i.e., by dividing i by each integer from 

2 to the square root of i, and checking if the remainder is 0. (There is no need to test 
beyond the square root of i, since if i has a divisor other than 1 and itself, it must 
also have a divisor that is no greater than its square root.) Since each test has time 
complexity <9( V^)/ the time complexity of the entire computation is upper bounded 
by 0(n X jn), i- e v 0{n 3/2 ). 

Intuitively, the brute-force algorithm tests each number from 1 to n independently, 
and does not exploit the fact that we need to compute all primes from It on. Heuris- 
tically, a better approach is to compute the primes and when a number is identified 
as a prime, to "sieve" it, i.e., remove all its multiples from future consideration. 

We use a Boolean array to encode the candidates, i.e., if the fth entry in the array 
is true, then i is potentially a prime. Initially, every number greater than or equal to 
2 is a candidate. Whenever we determine a number is a prime, we will add it to the 
result, which is an array. The first prime is 2. We add it to the result. None of its 
multiples can be primes, so remove all its multiples from the candidate set by writing 
false in the corresponding locations. The next location set to true is 3. It must be a 
prime since nothing smaller than it and greater than 1 is a divisor of it. As before, we 
add it to result and remove its multiples from the candidate array. We continue till 
we get to the end of the array of candidates. 

As an example, if n - 10, the candidate array is initialized to 

(F, F, T, T, T, T, T, T, T, T, T), where T is true and F is false. (Entries 0 and 1 are false, 
since 0 and 1 are not primes.) We begin with index 2. Since the corresponding en¬ 
try is one, we add 2 to the list of primes, and sieve out its multiples. The array is 
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now (F, F, T, T, F, T, F, T, F, T, F). The next nonzero entry is 3, so we add it to the list of 
primes, and sieve out its multiples. The array is now (F, F, T, T, F, T, F, T, F, F, F). The 
next nonzero entries are 5 and 7, and neither of them can be used to sieve out more 
entries. 


// Given n, return all primes up to and including n. 
public static List<Integer> generatePrimes(int n) { 

Listdnteger> primes = new ArrayList<>() ; 

// isPrime . get(p) represents if p is prime or not. Initially, set each 
// to true, excepting <9 and 1. Then use sieving to eliminate nonprimes. 
List<Boolean> isPrime = new ArrayList<>(Collections.nCopies(n + 1, true)); 
isPrime.set(®, false); 
isPrime.set(1, false); 
for (int p = 2; p <= n; ++p) { 
if (isPrime.get(p)) { 
primes.add(p); 

// Sieve p’s multiples. 
for (int j = p; j <= n; j += p) { 
isPrime.set(j, false); 

} 

} 

} 

return primes; 


We justified the sifting approach over the trial-division algorithm on heuristic 
grounds. The time to sift out the multiples of p is proportional to n/p, so the overall 
time complexity is 0(n/2 + n/3 + n/5 + n/7 + n /11 + ...). Although not obvious, this 
sum asymptotically tends to n log log n, yielding an <9(nloglogn) time bound. The 
space complexity is dominated by the storage for P, i.e., 0{n). 

The bound we gave for the trial-division approach, namely 0(n 3/2 ), is based on an 
0( yfn) bound for each individual test. Since most numbers are not prime, the actual 
time complexity of trial-division is actually lower on average, since the test frequently 
early-returns false. It is known that the time complexity of the trial-division approach 
is 0{n 3/2 / (log n) 2 ), so sieving is in fact superior to trial-division. 

We can improve runtime by sieving p's multiples from p 2 instead of p, since all 
numbers of the form kp, where k < p have already been sieved out. The storage can 
be reduced by ignoring even numbers. The code below reflects these optimizations. 


// Given n, return all primes up to and including n. 
public static List<Integer> generatePrimes(int n) { 
final int size = (int)Math.floor(®.5 * (n - 3)) + 1; 

List dnteger > primes = new ArrayList <>() ; 
primes.add(2) ; 

// isPrime . get(i) represents whether (2i + 3) is prime or not. 

// Initially, set each to true. Then use sieving to eliminate nonprimes. 

List<Boolean> isPrime = new ArrayList<>(Collections.nCopies(size, true)); 
for (int i = ®; i < size; ++i) { 
if (isPrime.get(i)) { 
int p = ((i * 2) + 3); 
primes.add(p); 

// Sieving from p A 2, whose value is (4i A 2 + 12i + 9). The index of this 
// value in isPrime is (2i A 2 + 6i + 3) because isPrime.get(i) represents 
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// 2i + 3. 

// 

// Note that we need to use long type for j because p A 2 might overflow. 
for (long j = ((i *i)*2)+6*i + 3;j< size; j += p) { 
isPrime.setC(int)j, false); 

} 

} 

} 

return primes; 


The asymptotic time and space complexity are the same as that for the basic sieving 
approach. 


6.9 Permute the elements of an array 

A permutation is a rearrangement of members of a sequence into a new sequence. For 
example, there are 24 permutations of (a, b, c, d); some of these are {b, a, d, c), (d, a, b, c), 
and {a, d, b, c). 

A permutation can be specified by an array P, where P[i] represents the location 
of the element at i in the permutation. For example, the array (2,0,1,3) represents 
the permutation that maps the element at location 0 to location 2, the element at 
location 1 to location 0, the element at location 2 to location 1, and keep the element 
at location 3 unchanged. A permutation can be applied to an array to reorder the 
array. For example, the permutation (2,0,1,3) applied to A = (a,b,c,d) yields the 
array (b, c, a, d). 

Given an array A of n elements and a permutation P, apply P to A. 

Hint: Any permutation can be viewed as a set of cyclic permutations. For an element in a cycle, 
how would you identify if it has been permuted? 

Solution: It is simple to apply a permutation-array to a given array if additional 
storage is available to write the resulting array. We allocate a new array B of the same 
length, set B[P[i]] = A[i] for each i, and then copy B to A. The time complexity is 0(n), 
and the additional space complexity is 0(n). 

A key insight to improving space complexity is to decompose permutations into 
simpler structures which can be processed incrementally. For example, consider the 
permutation (3,2,1,0). To apply it to an array A = (a, b, c, 6), we move the element at 
index 0 (a) to index 3 and the element already at index 3 (6) to index 0. Continuing, 
we move the element at index 1 (b) to index 2 and the element already at index 2 (c) 
to index 1. Now all elements have been moved according to the permutation, and 
the result is (6, c, b, a). 

This example generalizes: every permutation can be represented by a collection 
of independent permutations, each of which is cyclic , that is, it moves all elements by 
a fixed offset, wrapping around. 

This is significant, because a single cyclic permutation can be performed one ele¬ 
ment at a time, i.e., with constant additional storage. Consequently, if the permutation 
is described as a set of cyclic permutations, it can easily be applied using a constant 
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amount of additional storage by applying each cyclic permutation one-at-a-time. 
Therefore, we want to identify the disjoint cycles that constitute the permutation. 

To find and apply the cycle that includes entry i we just keep going forward (from i 
to P[i]) till we get back to i. After we are done with that cycle, we need to find another 
cycle that has not yet been applied. It is trivial to do this by storing a Boolean for each 
array element. 

One way to perform this without explicitly using additional 0(n) storage is to 
use the sign bit in the entries in the permutation-array. Specifically, we subtract n 
from P[i] after applying it. This means that if an entry in P[i] is negative, we have 
performed the corresponding move. 

For example, to apply (3,1,2,0}, we begin with the first entry, 3. We move A[0] 
to A[ 3], first saving the original A[ 3]. We update the permutation to (-1,1,2,0). We 
move A[ 3] to A[0]. Since P[ 0] is negative we know we are done with the cycle starting 
at 0. We also update the permutation to (-1,1,2, -4). Now we examine P[l]. Since it 
is not negative, it means the cycle it belongs to cannot have been applied. We continue 
as before. 

public static void applyPermutation(List<Integer> perm, Listdnteger> A) { 
for (int i = 8; i < A.sizeO; ++i) { 

// Check if the element at index i has not been moved by checking if 
// perm.get(i) is nonnegative. 
int next = i; 

while (perm.get(next) >= ®) { 

Collections.swap(A, i, perm.get(next)); 
int temp = perm.get(next); 

// Subtracts perm.size() from an entry in perm to make it negative, 

// which indicates the corresponding move has been performed. 
perm.set(next, perm.get(next) - perm.size()) ; 
next = temp; 

> 

} 

// Restore perm. 

for (int i = 8; i < perm.size(); i++) { 
perm.set(i, perm.get(i) + perm.size()); 

} 

} 


The program above will apply the permutation in 0(n) time. The space complexity is 
(9(1), assuming we can temporarily modify the sign bit from entries in the permutation 
array. 

If we cannot use the sign bit, we can allocate an array of n Booleans indicating 
whether the element at index i has been processed. Alternatively, we can avoid using 
0(n) additional storage by going from left-to-right and applying the cycle only if the 
current position is the leftmost position in the cycle. 

public static void applyPermutation(List dnteger > perm, List dnteger > A) { 
for (int i = 8; i < A.sizeO; ++i) { 

// Traverses the cycle to see if i is the minimum element. 
boolean isMin = true; 
int j = perm.get(i); 
while (j != i) { 
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if (j < i) { 
isMin. = false; 
break; 

} 

j = perm.get(j); 

} 

if (isMin) { 

cyclicPermutation(i, perm, A); 

} 

} 

} 

private static void cyclicPermutation(int start, Listdnteger> perm, 

List<Integer> A) { 

int i = start; 

int temp = A.get(start) ; 

do { 

int nextl = perm.get(i); 
int nextTemp = A.get(nextl); 

A.set(nextl, temp); 

i = nextl; 

temp = nextTemp; 

} while (i != start); 


Testing whether the current position is the leftmost position entails traversing the 
cycle once more, which increases the run time to 0(n 2 ). 

Variant: Given an array A of integers representing a permutation, update A to repre¬ 
sent the inverse permutation using only constant additional storage. 


6.10 Compute the next permutation 

There exist exactly n\ permutations of n elements. These can be totally ordered using 
the dictionary ordering —define permutation p to appear before permutation q if in the 
first place where p and q differ in their array representations, starting from index 0, the 
corresponding entry for p is less than that for q. For example, (2, 0,1) < (2,1,0). Note 
that the permutation (0,1,2) is the smallest permutation under dictionary ordering, 
and (2,1,0) is the largest permutation under dictionary ordering. 

Write a program that takes as input a permutation, and returns the next permutation 
under dictionary ordering. If the permutation is the last permutation, return the 
empty array. For example, if the input is (1,0,3,2) your function should return 
(1,2,0,3). If the input is (3,2,1,0), return (). 

Hint: Study concrete examples. 

Solution: A brute-force approach might be to find all permutations whose length 
equals that of the input array, sort them according to the dictionary order, then find 
the successor of the input permutation in that ordering. Apart from the enormous 
space and time complexity this entails, simply computing all permutations of length 
n is a nontrivial problem; see Problem 16.3 on Page 287 for details. 
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The key insight is that we want to increase the permutation by as little as possible. 
The loose analogy is how a car's odometer increments; the difference is that we cannot 
change values, only reorder them. We will use the permutation (6,2,1,5,4,3,0) to 
develop this approach. 

Specifically, we start from the right, and look at the longest decreasing suffix, which 
is (5,4,3,0) for our example. We cannot get the next permutation just by modifying 
this suffix, since it is already the maximum it can be. 

Instead we look at the entry e that appears just before the longest decreasing suffix, 
which is 1 in this case. (If there's no such element, i.e., the longest decreasing suffix 
is the entire permutation, the permutation must be (n - 1, n - 2,..., 2,1,0), for which 
there is no next permutation.) 

Observe that e must be less than some entries in the suffix (since the entry imme¬ 
diately after e is greater than e). Intuitively, we should swap e with the smallest entry 
s in the suffix which is larger than e so as to minimize the change to the prefix (which 
is defined to be the part of the sequence that appears before the suffix). 

For our example, e is 1 and s is 3. Swapping s and e results in (6,2,3,5,4,1,0). 

We are not done yet—the new prefix is the smallest possible for all permutations 
greater than the initial permutation, but the new suffix may not be the smallest. We 
can get the smallest suffix by sorting the entries in the suffix from smallest to largest. 
For our working example, this yields the suffix (0,1,4,5). 

As an optimization, it is not necessary to call a full blown sorting algorithm on 
suffix. Since the suffix was initially decreasing, and after replacing s by e it remains 
decreasing, reversing the suffix has the effect of sorting it from smallest to largest. 

The general algorithm for computing the next permutation is as follows: 

(1.) Find k such that p[k] < p[k + 1] and entries after index k appear in decreasing 
order. 

(2.) Find the smallest p[l] such that p[l] > p[k] (such an / must exist since p[k] < 
p[k+l]). 

(3.) Swap p[l] and p[k] (note that the sequence after position k remains in decreasing 
order). 

(4.) Reverse the sequence after position k. 

public static List<Integer> nextPermutation(List<Integer> perm) { 
int k = perm.sizeQ - 2; 

while (k >= Q && perm.get(k) >= perm.get(k + 1)) { 

--k; 

} 

if (k == -1) { 

return Collections.emptyList (); // perm is the last permutation. 

} 

// Swap the smallest entry after index k that is greater than perm[k ]. We 
// exploit the fact that perm.subList(k + 1, perm.size()) is decreasing so 
// if we search in reverse order, the first entry that is greater than 
// perm[k] is the smallest such entry. 
for (int i = perm.size() - 1; i > k; --i) { 
if (perm.get(i) > perm.get(k)) { 

Collections.swap(perm, k, i) ; 
break; 

} 
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} 

// Since perm. subList [k + 1, perm.sizeQ') is in decreasing order, we can 
// build the smallest dictionary ordering of this subarray by reversing it. 
Collections.reverse(perm.subList(k + 1, perm.size())); 
return perm; 


Each step is an iteration through an array, so the time complexity is 0(n). All that we 
use are a few local variables, so the additional space complexity is 0(1). 

Variant: Compute the A:th permutation under dictionary ordering, starting from the 
identity permutation (which is the first permutation in dictionary ordering). 

Variant: Given a permutation p, return the permutation corresponding to the previous 
permutation of p under dictionary ordering. 


6.11 Sample offline data 

This problem is motivated by the need for a company to select a random subset of 
its customers to roll out a new feature to. For example, a social networking company 
may want to see the effect of a new UI on page visit duration without taking the 
chance of alienating all its users if the rollout is unsuccessful. 

Implement an algorithm that takes as input an array of distinct elements and a size, 
and returns a subset of the given size of the array elements. All subsets should be 
equally likely. Return the result in input array itself. 

Hint: How would you construct a random subset of size k + 1 given a random subset of size k? 

Solution: Let the input array be A, its length n, and the specified size k. A naive 
approach is to iterate through the input array, selecting entries with probability k/n. 
Although the average number of selected entries is k, we may select more or less than 
k entries in this way. 

Another approach is to enumerate all subsets of size k and then select one at random 
from these. Since there are (”) subsets of size k, the time and space complexity are 
huge. Furthermore, enumerating all subsets of size k is nontrivial (Problem 16.5 on 
Page 291). 

The key to efficiently building a random subset of size exactly k is to first build one 
of size k- 1 and then adding one more element, selected randomly from the rest. The 
problem is trivial when k - 1. We make one call to the random number generator, 
take the returned value mod n (call it r), and swap A[ 0] with A[r]. The entry A[0] now 
holds the result. 

For k > 1, we begin by choosing one element at random as above and we now 
repeat the same process with the n — 1 element subarray A [1 : n - 1]. Eventually, the 
random subset occupies the slots A[0 : k - 1] and the remaining elements are in the 
last n-k slots. 

Intuitively, if all subsets of size k are equally likely, then the construction process 
ensures that the subsets of size k + 1 are also equally likely. A formal proof, which we 
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do not present, uses mathematical induction—the induction hypothesis is that every 
permutation of every size k subset of A is equally likely to be in A [0 :k — 1]. 

As a concrete example, let the input be A = (3,7,5,11) and the size be 3. In the 
first iteration, we use the random number generator to pick a random integer in the 
interval [0,3]. Let the returned random number be 2. We swap A[ 0] with A[ 2]— 
now the array is (5,7,3,11). Now we pick a random integer in the interval [1,3]. 
Let the returned random number be 3. We swap A[ 1] with A[ 3]—now the resulting 
array is (5,11,3,7). Now we pick a random integer in the interval [2,3]. Let the 
returned random number be 2. When we swap A[ 2] with itself the resulting array is 
unchanged. The random subset consists of the first three entries, i.e., [5,11,3}. 

public static void randomSampling(int k, Listdnteger> A) { 

Random gen = new Random(); 
for (int i = Q; i < k; ++i) { 

// Generate a random int in [i, A.size() - 1]. 

Collections.swap(A, i, i + gen.nextlnt(A.size() - i)); 

} 

} 


The algorithm clearly runs in additional <9(1) space. The time complexity is 0(k) to 
select the elements. 

The algorithm makes k calls to the random number generator. When k is bigger 
than |, we can optimize by computing a subset of n-k elements to remove from the 
set. For example, when k = n - 1, this replaces n - 1 calls to the random number 
generator with a single call. 

Variant: The rand() function in the standard C library returns a uniformly random 
number in [0, RAND_MAX - 1]. Does rand() mod n generate a number uniformly dis¬ 
tributed in [0, n - 1]? 

6.12 Sample online data 

This problem is motivated by the design of a packet sniffer that provides a uniform 
sample of packets for a network session. 

Design a program that takes as input a size k, and reads packets, continuously main¬ 
taining a uniform random subset of size k of the read packets. 

Hint: Suppose you have a procedure which selects k packets from the first n > k packets as 
specified. How would you deal with the (n + l)th packet? 

Solution: A brute force approach would be to store all the packets read so far. After 
reading in each packet, we apply Solution 6.11 on the facing page to compute a 
random subset of k packets. The space complexity is high— 0(n), after n packets 
have been read. The time complexity is also high— 0(nk), since each packet read is 
followed by a call to Solution 6.11 on the preceding page. 

At first glance it may seem that it is impossible to do better than the brute-force 
approach, since after reading the nth packet, we need to choose k packets uniformly 
from this set. However, suppose we have read the first n packets, and have a random 
subset of k of them. When we read the (n + l)th packet, it should belong to the new 
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subset with probability k/(n + 1). If we choose one of the packets in the existing subset 
uniformly randomly to remove, the resulting collection will be a random subset of 
the n + 1 packets. 

The formal proof that the algorithm works correctly, uses induction on the number 
of packets that have been read. Specifically, the induction hypothesis is that all A:-sized 
subsets are equally likely after n >k packets have been read. 

As an example, suppose k = 2, and the packets are read in the order p, q, r, t, u, v. 
We keep the first two packets in the subset, which is {p, q}. We select the next packet, r, 
with probability 2/3. Suppose it is not selected. Then the subset after reading the first 
three packets is still {p, q}. We select the next packet, t, with probability 2/4. Suppose 
it is selected. Then we choose one of the packets in {p, q] uniformly, and replace it 
with t. Let q be the selected packet—now the subset is {p, t\. We select the next packet 
u with probability 2/5. Suppose it is selected. Then we choose one of the packets in 
{p, t } uniformly, and replace it with u. Let t be the selected packet—now the subset is 
{p, u }. We select the next packet v with probability 2/6. Suppose it is not selected. The 
random subset remains {p, u). 


// Assumption: there are at least k elements in the stream. 

public static List<Integer> onlineRandomSample(Iterator<Integer> sequence, 

int k) { 

List<Integer> runningSample = new ArrayList<>(k); 

// Stores the first k elements. 

for (int i = Q; sequence.hasNext() && i < k; ++i) { 
runningSample.add(sequence.next()); 

} 

// Have read the first k elements. 
int numSeenSoFar = k; 

Random randldxGen = new Random(); 
while (sequence.hasNext()) { 

Integer x = sequence.next(); 

++numSeenSoFar; 

// Generate a random number in [9, numSeenSoFar], and if this number is in 
// [9, k - 1], we replace that element from the sample with x. 
final int idxToReplace = randldxGen.nextlnt(numSeenSoFar); 
if (idxToReplace < k) { 

runningSample.set(idxToReplace, x); 

} 

} 

return runningSample; 


The time complexity is proportional to the number of elements in the stream, since 
we spend 0(1) time per element. The space complexity is 0(k). 

Note that at each iteration, every subset is equally likely. However, the subsets are 
not independent from iteration to iteration—successive subsets differ in at most one 
element. In contrast, the subsets computed by brute-force algorithm are independent 
from iteration to iteration. 


80 



6.13 Compute a random permutation 


Generating random permutations is not as straightforward as it seems. For example, 
iterating through (0,1,..., n - 1) and swapping each element with another randomly 
selected element does not generate all permutations with equal probability. One way 
to see this is to consider the case n = 3. The number of permutations is 3! = 6. The 
total number of ways in which we can choose the elements to swap is 3 3 = 27 and 
all are equally likely. Since 27 is not divisible by 6, some permutations correspond to 
more ways than others, so not all permutations are equally likely. 

Design an algorithm that creates uniformly random permutations of {0,1,1}. 
You are given a random number generator that returns integers in the set {0,1,..., n-1) 
with equal probability; use as few calls to it as possible. 

Hint: If the result is stored in A, how would you proceed once A[n - 1] is assigned correctly? 

Solution: A brute-force approach might be to iteratively pick random numbers be¬ 
tween 0 and n-1, inclusive. If number repeats, we discard it, and try again. A hash 
table is a good way to store and test values that have already been picked. 

For example, if n = 4, we might have the sequence 1,2,1 (repeats), 3,1 (repeats), 2 
(repeat), 0 (done, all numbers from 0 to 3 are present). The corresponding permutation 
is <1,2,3,0). 

It is fairly clear that all permutations are equally likely with this approach. The 
space complexity beyond that of the result array is 0(n) for the hash table. The time 
complexity is slightly challenging to analyze. Early on, it takes very few iterations to 
get more new values, but it takes a long time to collect the last few values. Computing 
the average number of tries to complete the permutation in this way is known as the 
Coupon Collector's Problem. It is known that the number of tries on average (and 
hence the average time complexity) is 0(n log n). 

Clearly, the way to improve time complexity is to avoid repeats. We can do this 
by restricting the set we randomly choose the remaining values from. If we apply 
Solution 6.11 on Page 78 to <0,1,2,..., n - 1) with k = n, at each iteration the array is 
partitioned into the partial permutation and remaining values. Although the subset 
that is returned is unique (it will be {0, 1 ,..., n - 1}), all n\ possible orderings of the 
elements in the set occur with equal probability. For example, let n = 4. We begin with 
<0,1,2,3). The first random number is chosen between 0 and 3, inclusive. Suppose it 
is 1. We update the array to <1,0,2,3). The second random number is chosen between 
1 and 3, inclusive. Suppose it is 3. We update the array to <1,3,0,2). The third random 
number is chosen between 2 and 3, inclusive. Suppose it is 3. We update the array to 
<1,3,2,0). This is the returned result. 

public static List<Integer> computeRandomPermutation(int n) { 

List<Integer> permutation = new ArrayList<>(n); 
for (int i = ®; i < n; ++i) { 
permutation.add(i); 

} 

OfflineSampling.randomSampling(permutation.size(), permutation); 
return permutation; 

} 
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The time complexity is 0{n), and, as an added bonus, no storage outside of that 
needed for the permutation array itself is needed. 


6.14 Compute a random subset 

The set {0,1,2,..., n - 1} has (”) = n\/((n - k)\k\) subsets of size k. We seek to design 
an algorithm that returns any one of these subsets with equal probability. 

Write a program that takes as input a positive integer n and a size k <n, and returns 
a size-k subset of {0,1,2,..., n - 1}. The subset should be represented as an array. All 
subsets should be equally likely and, in addition, all permutations of elements of the 
array should be equally likely. You may assume you have a function which takes as 
input a nonnegative integer t and returns an integer in the set {0, 1 ,..., t - 1} with 
uniform probability. 

Hint: Simulate Solution 6.11 on Page 78, using an appropriate data structure to reduce space. 

Solution: Similar to the brute-force algorithm presented in Solution 6.13 on the 
preceding page, we could iteratively choose random numbers between 0 and n - 1 
until we get k distinct values. This approach suffers from the same performance 
degradation when k is close to n, and it also requires 0(k) additional space. 

We could mimic the offline sampling algorithm described in Solution 6.11 on 
Page 78, with A[i] = i initially, stopping after k iterations. This requires 0(n) space 
and 0(n) time to create the array. After creating (0,1,2,..., n - 1), we need 0(k) time 
to produce the subset. 

Note that when k <sc n, most of the array is untouched, i.e., A[i] = i. The key to 
reducing the space complexity to 0(k) is simulating A with a hash table. We do this 
by only tracking entries whose values are modified by the algorithm—the remainder 
have the default value, i.e., the value of an entry is its index. 

Specifically, we maintain a hash table H whose keys and values are from 
{0,1,1}. Conceptually, H tracks entries of the array which have been touched 
in the process of randomization—these are entries A[i] which may not equal i. The 
hash table H is updated as the algorithm advances. 

• If i is in H, then its value in H is the value stored at A[i] in the brute-force 
algorithm. 

• If i is not in H, then this implicitly implies A[i] = i. 

Since we track no more than k entries, when k is small compared to n, we save time 
and space over the brute-force approach, which has to initialize and update an array 
of length n. 

Initially, H is empty. We do k iterations of the following. Choose a random integer 
r in [0, n - 1 - z], where i is the current iteration count, starting at 0. There are four 
possibilities, corresponding to whether the two entries in A that are being swapped 
are already present or not present in H. The desired result is in A [0 :k — 1], which can 
be determined from H. 

For example, suppose n = 100 and k- 4. In the first iteration, suppose we get the 
random number 28. We update H to (0,28), (28,0). This means that A[0] is 28 and 
A[28] is 0—for all other i, A[i] = i. In the second iteration, suppose we get the random 
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number 42. We update H to (0,28), (28,0), (1,42), (42,1). In the third iteration, suppose 
we get the random number 28 again. We update H to (0,28), (28,2), (1,42), (42,1), (2,0). 
In the third iteration, suppose we get the random number 64. We update H to 
(0,28), (28,2), (1,42), (42,1), (2,0), (3,64), (64,3). The random subset is the 4 elements 
corresponding to indices 0,1,2,3, i.e., (28,42,0,64). 

// Returns a random k-sized subset of {<9, 1, n - 1}. 

public static List<Integer> randomSubset(int n, int k) { 

Mapdnteger , Integer> changedElements = new HashMap<>(); 

Random randldxGen = new RandomO ; 
for (int i = ®; i < k; ++i) { 

// Generate random int in [i, n - 1]. 

int randldx = i + randldxGen.nextlnt(n - i); 

Integer ptrl = changedElements.get(randldx); 

Integer ptr2 = changedElements.get(i); 
if (ptrl == null && ptr2 == null) { 
changedElements.put(randldx, i); 
changedElements.put(i, randldx); 

} else if (ptrl == null && ptr2 != null) { 
changedElements.put(randldx, ptr2); 
changedElements.put(i, randldx); 

} else if (ptrl != null && ptr2 == null) { 
changedElements.put(i, ptrl); 
changedElements.put(randldx, i); 

} else { 

changedElements.put(i, ptrl); 
changedElements.put(randldx, ptr2); 

} 


List<Integer> result = new ArrayList<>(k); 
for (int i = Q; i < k; ++i) { 

result.add(changedElements.get(i)) ; 

} 

return result; 


The time complexity is <9(/c), since we perform a bounded number of operations per 
iteration. The space complexity is also 0(k), since H and the result array never contain 
more than k entries. 


6.15 Generate nonuniform random numbers 

Suppose you need to write a load test for a server. You have studied the inter-arrival 
time of requests to the server over a period of one year. From this data you have 
computed a histogram of the distribution of the inter-arrival time of requests. In the 
load test you would like to generate requests for the server such that the inter-arrival 
times come from the same distribution that was observed in the historical data. The 
following problem formalizes the generation of inter-arrival times. 

You are given n numbers as well as probabilities po,p \,.. - ,p n -\, which sum up to 
1. Given a random number generator that produces values in [0,1] uniformly, how 
would you generate one of the n numbers according to the specified probabilities? For 
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example, if the numbers are 3,5,7,11, and the probabilities are 9/18,6/18,2/18,1/18, 
then in 1000000 calls to your program, 3 should appear roughly 500000 times, 5 should 
appear roughly 333333 times, 7 should appear roughly 111111 times, and 11 should 
appear roughly 55555 times. 

Hint: Look at the graph of the probability that the selected number is less than or equal to a. 
What do the jumps correspond to? 

Solution: First note that actual values of the numbers is immaterial—we want to 
choose from one of n outcomes with probabilities p 0 ,pi,.. . ,p„_ 1 - If all probabilities 
were the same, i.e., 1/n, we could make a single call to the random number generator, 
and choose outcome i if the number falls lies between i/n and (i + 1 )/n. 

For the case where the probabilities are not the same, we can solve the problem 
by partitioning the unit interval [0,1] into n disjoint segments, in a way so that the 
length of the jth interval is proportional to pj. Then we select a number uniformly 
at random in the unit interval, [0,1], and return the number corresponding to the 
interval the randomly generated number falls in. 

An easy way to create these intervals is to use p 0 ,p 0 + P\,Po + Pi + Pi, • • • ,Po + pi + 

p 2 +-1- p n -i as the endpoints. Using the example given in the problem statement, the 

four intervals are [0.0,0.5), [0.5,0.833), [0.833,0.944), [0.944,1.0]. Now, for example, 
if the random number generated uniformly in [0.0,1.0] is 0.873, since 0.873 lies in 
[0.833,0.944), which is the third interval, we return the third number, which is 7. 

In general, searching an array of n disjoint intervals for the interval containing a 
number takes 0(n) time. However, we can do better. Since the array (p 0 ,po + Pi,po + 
pi + pi, •. • ,po + pi + pi + * * * + Pn-i) is sorted, we can use binary search to find the 
interval in <9(log n) time. 


public static int nonuniformRandomNumberGeneration( 

List<Integer> values, List<Double> probabilities) { 

List<Double> prefixSumOfProbabilities = new ArrayList<>(); 
prefixSumOfProbabilities.add(®.®) ; 

// Creating the endpoints for the intervals corresponding to the 
// probabilities. 
for (double p : probabilities) { 
prefixSumOfProbabilities.add( 

prefixSumOfProbabilities.get(prefixSumOfProbabilities.size() - 1) 
+ P) ; 

} 

Random r = new Random(); 

// Get a random number in [9. 9 , 1. 9) . 
final double uniformOl = r.nextDouble(); 

// Find the index of the interval that uniformQl lies in. 

int it = Collections.binarySearch(prefixSumOfProbabilities, uniform®1); 

if (it < ®) { 

// Fife want the index of the first element in the array which is 
// greater than the key. 

// 

// When a key is not present in the array, Collections.binarySearch() 
// returns the negative of 1 plus the smallest index whose entry 
// is greater than the key. 

// 
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// Therefore, if the return value is negative, by taking its absolute 
// value and adding 1 to it, we get the desired index. 
final int intervalldx = (Math.abs(it) - 1) - 1; 
return values.get(intervalldx); 

} else { 

// Fife have it >= <9, i.e., uniformQl equals an entry 
// in prefixSumOfProbabilities. 

// 

// Because we uniformQl is a random double, the probability of it 
// equalling an endpoint in prefixSumOfProbabilities is exceedingly low. 
// However, it is not 19, so to be robust we must consider this case. 
return values.get(it) ; 

} 

} 


The time complexity to compute a single value is 0(n), which is the time to create the 
array of intervals. This array also implies an 0(n) space complexity. 

Once the array is constructed, computing each additional result entails one call to 
the uniform random number generator, followed by a binary search, i.e., <9(log n). 

Variant: Given a random number generator that produces values in [0,1] uniformly, 
how would you generate a value X from T according to a continuous probability 
distribution, such as the exponential distribution? 

Multidimensional arrays 

Thus far we have focused our attention in this chapter on one-dimensional arrays. 
We now turn our attention to multidimensional arrays. A 2D array in an array whose 
entries are themselves arrays; the concept generalizes naturally to k dimensional 
arrays. 

Multidimensional arrays arise in image processing, board games, graphs, mod¬ 
eling spatial phenomenon, etc. Often, but not always, the arrays that constitute the 
entries of a 2D array A have the same length, in which case we refer to A as being an 
mxn rectangular array (or sometimes just an m X n array), where m is the number of 
entries in A, and n the number of entries in A[0]. The elements within a 2D array A 
are often referred to by their row and column indices i and j, and written as A [/][/]. 

6.16 The Sudoku checker problem 

Sudoku is a popular logic-based combinatorial number placement puzzle. The objec¬ 
tive is to fill a 9x 9 grid with digits subject to the constraint that each column, each row, 
and each of the nine 3x3 sub-grids that compose the grid contains unique integers 
in [1,9]. The grid is initialized with a partial assignment as shown in Figure 6.2(a) on 
the following page; a complete solution is shown in Figure 6.2(b) on the next page. 

Check whether a 9 X 9 2D array representing a partially completed Sudoku is valid. 
Specifically, check that no row, column, or 3 X 3 2D subarray contains duplicates. A 
0-value in the 2D array indicates that entry is blank; every other entry is in [1,9]. 

Hint: Directly test the constraints. Use an array to encode sets. 
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(a) Partial assignment. (b) A complete solution. 

Figure 6.2: Sudoku configurations. 


Solution: There is no real scope for algorithm optimization in this problem—it's all 
about writing clean code. 

We need to check nine row constraints, nine column constraints, and nine sub-grid 
constraints. It is convenient to use bit arrays to test for constraint violations, that is 
to ensure no number in [1,9] appears more than once. 


// Check if a partially filled matrix has any conflicts. 

public static boolean isValidSudoku(List<List<Integer>> partialAssignment) { 
// Check row constraints. 

for (int i = 8; i < partialAssignment.size(); ++i) { 
if (hasDuplicate(partialAssignment, i, i + 1, 8, 
partialAssignment.size())) { 

return false; 

} 

} 

// Check column constraints. 

for (int j = 8; j < partialAssignment.size(); ++j) { 

if (hasDuplicate(partialAssignment , 8 , partialAssignment . size() , j, 

j + D) { 

return false; 

} 

} 

// Check region constraints. 

int regionSize = (int)Math.sqrt(partialAssignment.size()); 
for (int I = 8; I < regionSize; ++I) { 
for (int J = 8; J < regionSize; ++J) { 

if (hasDuplicate(partialAssignment, regionSize * I, 

regionSize * (I + 1), regionSize * J, 
regionSize * (J + 1))) { 

return false; 

} 

} 

} 

return true; 
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// Return true if subarray partialAssignment[startRow : endRow - 1][startCol : 
// endCol - 1] contains any duplicates in {1, 2, 

// partialAssignment . size()}; otherwise return false. 

private static boolean hasDuplicate(List<List<Integer>> partialAssignment, 

int startRow, int endRow, int startCol, 
int endCol) { 

List<Boolean> isPresent = new ArrayList<>( 

Collections.nCopies(partialAssignment.size() + 1, false)); 
for (int i = startRow; i < endRow; ++i) { 
for (int j = startCol; j < endCol; ++j) { 
if (partialAssignment.get(i).get(j) != ® 

&& isPresent.get(partialAssignment.get(i).get(j))) { 
return true; 

} 

isPresent.set(partialAssignment.get(i).get(j) , true); 

} 

} 

return false; 

} 

The time complexity of this algorithm for an nxn Sudoku grid with yfnx fn subgrids 
is0(n 2 )+0(n 2 )+0(n 2 /( yfn) 2 x{ yfn) 2 ) = 0(n 2 )} the terms correspond to the complexity 
to check n row constraints, the n column constraints, and the n subgrid constraints, 
respectively. The memory usage is dominated by the bit array used to check the 
constraints, so the space complexity is 0(n). 

Solution 16.9 on Page 296 describes how to solve Sudoku instances. 

6.17 Compute the spiral ordering of a 2D array 

A 2D array can be written as a sequence in several orders—the most natural ones 
being row-by-row or column-by-column. In this problem we explore the problem 
of writing the 2D array in spiral order. For example, the spiral ordering for the 2D 
array in Figure 6.3(a) is (1,2,3,6,9,8,7,4,5). For Figure 6.3(b), the spiral ordering is 
<1,2,3,4,8,12,16,15,14,13,9,5,6,7,11,10). 


CO Cl C2 R0 

R0 1 2 3 R1 

R1 4 5 6 R2 

R2 7 8 9 R3 

(a) Spiral ordering for a 3 x 3 (b) Spiral ordering for a 4 x 4 array, 

array. 

Figure 6.3: Spiral orderings. Column and row ids are specified above and to the left of the matrix, i.e., 
1 is at entry (0,0). 
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Write a program which takes an n X n 2D array and returns the spiral ordering of the 
array. 

Hint: Use case analysis and divide-and-conquer. 

Solution: It is natural to solve this problem starting from the outside, and working 
to the center. The naive approach begins by adding the first row, which consists of n 
elements. Next we add the n — 1 remaining elements of the last column, then the n — 1 
remaining elements of the last row, and then the n — 2 remaining elements of the first 
column. The lack of uniformity makes it hard to get the code right. 

Here is a uniform way of adding the boundary. Add the first n — 1 elements of the 
first row. Then add the first n — 1 elements of the last column. Then add the last n — 1 
elements of the last row in reverse order. Finally, add the last n — 1 elements of the 
first column in reverse order. 

After this, we are left with the problem of adding the elements of an (n - 2) X (n - 2) 
2D array in spiral order. This leads to an iterative algorithm that adds the outermost 
elements of n x n, (n - 2) X (n - 2), (n - 4) x (n - 4),... 2D arrays. Note that a matrix 
of odd dimension has a comer-case, namely when we reach its center. 

As an example, for the 3x3 array in Figure 6.3(a) on the previous page, we would 
add 1,2 (first two elements of the first row), then 3,6 (first two elements of the last 
column), then 9,8 (last two elements of the last row), then 7,4 (last two elements of 
the first column). We are now left with the lxl array, whose sole element is 5. After 
processing it, all elements are processed. 

For the 4x4 array in Figure 6.3(b) on the preceding page, we would add 1,2,3 (first 
three elements of the first row), then 4,8,12 (first three elements of the last column), 
then 16,15,14 (last three elements of the last row), then 13,9,5 (last three elements of 
the first column). We are now left with a 2 X 2 matrix, which we process similarly in 
the order 6,7,11,10, after which all elements are processed. 

public static List<Integer> matrixInSpiralOrder( 

List<List<Integer» squareMatrix) { 

List<Integer> spiralOrdering = new ArrayList<>(); 

for (int offset = Q; offset < Math.ceil(®.5 * squareMatrix.size()); 

++offset) { 

matrixLayerlnClockwise(squareMatrix, offset, spiralOrdering); 

} 

return spiralOrdering; 

} 

private static void matrixLayerlnClockwise(List<List<Integer>> squareMatrix, 

int offset, 

Listdnteger> spiralOrdering) { 
if (offset == squareMatrix.size() - offset - 1) { 

// squareMatrix has odd dimension , and we are at its center. 
spiralOrdering.add(squareMatrix.get(offset).get(offset)); 

return; 

} 

for (int j = offset; j < squareMatrix.size() - offset - 1; ++j ) { 
spiralOrdering.add(squareMatrix.get(offset).get(j)); 

} 

for (int i = offset; i < squareMatrix.size() - offset - 1; ++i) { 
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spiralOrdering.add( 

squareMatrix.get(i).get(squareMatrix.size() - offset - 1)); 

} 

for (int j = squareMatrix.size() - offset - 1; j > offset; --j) { 
spiralOrdering.add( 

squareMatrix.get(squareMatrix.size() - offset - l).get(j)); 

} 

for (int i = squareMatrix.size() - offset - 1; i > offset; --i) { 
spiralOrdering.add(squareMatrix.get(i).get(offset)); 

} 

} 


The time complexity is 0(n 2 ) and the space complexity is 0(1). 

The above solution uses four iterations which are almost identical. Now we present 
a solution that uses a single iteration that tracks the next element to process and the 
direction—left,right,up,down—to advance in. Think of the matrix as laid out on a 2D 
grid with X- and Y-axes. The pair (i, j) denotes the entry in Column i and Row j. Let 
(x, y) be the next element to process. Initially we move to the right (incrementing x 
until (n -1, 0) is processed). Then we move down (incrementing y until (n - 1, n -1) is 
processed). Then we move left (decrementing x until (0, n - 1) is processed). Then we 
move up (decrementing y until (0,1) is processed). Note that we stop at 1, not 0, since 
(0,0) was already processed. We record that an element has already been processed 
by setting it to 0, which is assumed to be a value that is not already present in the 
array. (Any value not in the array works too.) After processing (0,1) we move to the 
right till we get to (n - 2,1) (since (n - 2,0) was already processed). This method is 
applied until all elements are processed. 

public static List<Integer> matrixInSpiralOrder( 

List<List<Integer» squareMatrix) { 

final int[][] SHIFT = {{©, 1}, {1, ®} , {®, -1}, {-1, ®}}; 

int dir = ®, x = ®, y = ®; 

List<Integer> spiralOrdering = new ArrayList<>(); 

for (int i = ®; i < squareMatrix.size() * squareMatrix.size(); ++i) { 
spiralOrdering.add(squareMatrix.get(x).get(y)); 
squareMatrix.get(x).set(y, ®); 

int nextX = x + SHIFT[dir][®], nextY = y + SHIFT[dir][1]; 
if (nextX < ® || nextX >= squareMatrix.size() || nextY < ® 

|| nextY >= squareMatrix.size() 

|| squareMatrix.get(nextX).get(nextY) == ®) { 
dir = (dir + 1) % 4; 
nextX = x + SHIFT[dir] [®] ; 
nextY = y + SHIFT[dir][1]; 

} 

x = nextX; 
y = nextY; 

} 

return spiralOrdering; 

} 


The time complexity is 0(n 2 ) and the space complexity is 0(1). 

Variant: Given a dimension d, write a program to generate a dx d 2D array which in 
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spiral order is (1,2,3 ,d 2 ). For example, if d = 3, the result should be 


A = 


1 

8 

7 


2 

9 

6 


3 

4 

5 


Variant: Given a sequence of integers P, compute a 2D array A whose spiral order is 
P. (Assume the size of P is n 2 for some integer n.) 

Variant: Write a program to enumerate the first n pairs of integers (a, b) in spiral order, 
starting from (0,0) followed by (1,0). For example, if n - 10, your output should be 
(0,0), (1,0), (1, -1), (0, -1), (-1, -1), (-1,0), (-1,1), (0,1), (1,1), (2,1). 

Variant: Compute the spiral order for an mXn 2D array A. 

Variant: Compute the last element in spiral order for an m X n 2D array A in (9(1) 
time. 

Variant: Compute the kth element in spiral order for an m x n 2D array A in (9(1) time. 
6.18 Rotate a 2D array 

Image rotation is a fundamental operation in computer graphics. Figure 6.4 illustrates 
the rotation operation on a 2D array representing a bit-map of an image. Specifically, 
the image is rotated by 90 degrees clockwise. 
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(a) Initial 4 x 4 2D array. (b) Array rotated by 90 degrees clockwise. 
Figure 6.4: Example of 2D array rotation. 


Write a function that takes as input an n X n 2D array, and rotates the array by 
90 degrees clockwise. 

Hint: Focus on the boundary elements. 

Solution: With a little experimentation, it is easy to see that ith column of the rotated 
matrix is the ith row of the original matrix. For example, the first row, (13,14,15,16) 
of the initial array in 6.4 becomes the first column in the rotated version. Therefore, a 
brute-force approach is to allocate a new nXn 2D array, write the rotation to it (writing 
rows of the original matrix into the columns of the new matrix), and then copying the 
new array back to the original one. The last step is needed since the problem says to 
update the original array. The time and additional space complexity are both 0(n 2 ). 
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Since we are not explicitly required to allocate a new array, it is natural to ask if we 
can perform the rotation in-place, i.e., with 0(1) additional storage. The first insight 
is that we can perform the rotation in a layer-by-layer fashion—different layers can 
be processed independently. Furthermore, within a layer, we can exchange groups 
of four elements at a time to perform the rotation, e.g., send 1 to 4's location, 4 to 16's 
location, 16 to 13's location, and 13 to l's location, then send 2 to 8's location, 8 to 
15's location, 15 to 9's location, and 9 to 2's location, etc. The program below works 
its way into the center of the array from the outermost layers, performing exchanges 
within a layer iteratively using the four-way swap just described. 

public static void rotateMatrix(List<List<Integer>> squareMatrix) { 
final int matrixSize = squareMatrix.size() - 1; 
for (int i = ®; i < (squareMatrix.size() / 2); ++i) { 
for (int j = i; j < matrixSize - i; ++j) { 

// Perform a 4-way exchange. 

int tempi = squareMatrix.get(matrixSize - j).get(i); 

int temp2 = squareMatrix.get(matrixSize - i).get(matrixSize - j); 

int temp3 = squareMatrix.get(j).get(matrixSize - i); 

int temp4 = squareMatrix.get(i) . get (j) ; 

squareMatrix.get(i).set(j, tempi); 

squareMatrix.get(matrixSize - j).set(i, temp2); 

squareMatrix.get(matrixSize - i).set(matrixSize - j, temp3); 

squareMatrix.get(j).set(matrixSize - i, temp4); 

} 

} 

} 


The time complexity is 0(n 2 ) and the additional space complexity is 0(1). 

Interestingly, we can get the effect of a rotation with 0(1) space and time complexity, 
albeit with some limitations. Specifically, we return an object r that composes the 
original matrix A. A read of the element at indices i and ; in r is converted into a read 
from A at index [n - 1 - j][i]. Writes are handled similarly. The time to create r is 
constant, since it simply consists of a reference to A. The time to perform reads and 
writes is unchanged. This approach breaks when there are clients of the original A 
object, since writes to r change A. Even if A is not written to, if methods on the stored 
objects change their state, the system gets corrupted. Copy-on-write can be used to 
solve these issues. 

class RotatedMatrix { 

private List<List<Integer» wrappedSquareMatrix; 

public RotatedMatrix(List<List<Integer>> squareMatrix) { 
this.wrappedSquareMatrix = squareMatrix; 

} 

public int readEntry(int i, int j) { 

return wrappedSquareMatrix.get(wrappedSquareMatrix.size() - 1 - j).get(i); 

} 

public void writeEntry(int i, int j, int v) { 

wrappedSquareMatrix.get(wrappedSquareMatrix.size() - 1 - j).set(i, v) ; 

} 
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Figure 6.5: A Pascal triangle. 


Variant: Implement an algorithm to reflect A, assumed to be an n X n 2D array, about 
the horizontal axis of symmetry. Repeat the same for reflections about the vertical 
axis, the diagonal from top-left to bottom-right, and the diagonal from top-right to 
bottom-left. 


6.19 Compute rows in Pascal's Triangle 

Figure 6.5 shows the first five rows of a graphic that is known as Pascal's triangle. 
Each row contains one more entry than the previous one. Except for entries in the last 
row, each entry is adjacent to one or two numbers in the row below it. The first row 
holds 1. Each entry holds the sum of the numbers in the adjacent entries above it. 

Write a program which takes as input a nonnegative integer n and returns the first n 
rows of Pascal's triangle. 

Hint: Write the given fact as an equation. 

Solution: A brute-force approach might be to organize the arrays in memory similar 
to how they appear in the figure. The challenge is to determine the correct indices to 
range over and to read from. 

A better approach is to keep the arrays left-aligned, that is the first entry is at 
location 0. Now it is simple: the ;th entry in the ith row is 1 if / = 0 or j = i, otherwise 
it is the sum of the (j - l)th and ;th entries in the (i - l)th row. The first row R 0 is (1). 
The second row R\ is (1,1). The third row R 2 is (l,Ri[0] + Ri[l] = 2,1). The fourth 
row R 3 is (1, R 2 [0] + R 2 [l] = 3,R 2 [1] + £ 2 [2] = 3,1). 

public static List<List<Integer» generatePascalTriangle(int numRows) { 

List<List<Integer>> pascalTriangle = new ArrayList<>(); 
for (int i = Q; i < numRows; ++i) { 

Listdnteger> currRow = new ArrayList<>() ; 
for (int j = ®; j <= i; ++j) { 

// Set this entry to the sum of the two above adjacent entries if they 
// exist. 

currRow.add(C® < j && j < i) 

? pascalTriangle.get(i - l).get(j - 1) 

+ pascalTriangle.get(i - l).get(j) 

: l); 

} 

pascalTriangle.add(currRow); 

} 
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return pascalTriangle; 


} 


Since each element takes (9(1) time to compute, the time complexity is 0(1 +2+— \-n) = 
0(n(n + l)/2) = 0(n 2 ). Similarly, the space complexity is 0(n 2 ). 

It is a fact that the ith entry in the nth row of Pascal's triangle is (”). This in itself 
does not trivialize the problem, since computing (”) itself is tricky. (In fact, Pascal's 
triangle can be used to compute (").) 

Variant: Compute the nth row of Pascal's triangle using 0(n) space. 
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Chapter 


I Strings 

String pattern matching is an important problem that occurs in 
many areas of science and information processing. In computing, 
it occurs naturally as part of data processing, text editing, term 
rewriting, lexical analysis, and information retrieval. 

— "Algorithms For Finding Patterns in Strings," 
A. V. Aho, 1990 

Strings are ubiquitous in programming today—scripting, web development, and 
bioinformatics all make extensive use of strings. You should know how strings are 
represented in memory, and understand basic operations on strings such as compar¬ 
ison, copying, joining, splitting, matching, etc. We now present problems on strings 
which can be solved using elementary techniques. Advanced string processing algo¬ 
rithms often use hash tables (Chapter 13) and dynamic programming (Page 303). 

Strings boot camp 

A palindromic string is one which reads the same when it is reversed. The program 
below checks whether a string is palindromic. Rather than creating a new string for 
the reverse of the input string, it traverses the input string forwards and backwards, 
thereby saving space. Notice how it uniformly handles even and odd length strings. 

public static boolean isPalindromic(String s) { 

for (int i = Q, j = s.lengthO - 1; i < j; ++i, --j) { 
if (s.charAt(i) != s.charAt(j)) { 

return false; 

} 

} 

return true; 

} 

The time complexity is 0(n) and the space complexity is (9(1), where n is the length 
of the string. 

Know your string libraries 

When manipulating Java strings, you need to know both the String class as well as 
the StringBuilder class. The latter is needed because Java strings are immutable, so 
to make string construction efficient, it's necessary to have a mutable string class. 

• The key methods on strings are char At (1), compareT o ( “ f oo” ), concat ( “bar” ) 
(returns a new string—does not update the invoking string), contains ( “aba” ), 
endsWith(“YZ”), indexOf(“needle”), indexOf(“needle”, 12), 
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Similar to arrays, string problems often have simple brute-force solutions that use 
0(n) space solution, but subtler solutions that use the string itself to reduce space 
complexity to 0(1). [Problems 7.6 and 7.4] 

Understand the implications of a string type which is immutable, e.g., the need to 
allocate a new string when concatenating immutable strings. Know alternatives 
to immutable strings, e.g., an array of characters or a StringBuilder in Java. 
[Problem 7.6] 

Updating a mutable string from the front is slow, so see if it's possible to write 
values from the back. [Problem 7.4] 


indexOf( ’ A’) ,, indexOf(’B’, offset) ,, lastlndexOf(“needle”), lengthO, 
replace(’a’, ’A’), replace(“a” “ABC”), “foo::bar::abc”.split(“::”), 
startsWith(prefix), startsWith(“www” , “http: //” . lengthO), 

substring(l), substringCl, 5), toCharArray(), toLowerCaseO, and trim(). 
- The substring() method is particularly important, and also easy to get 
wrong, since it has two variants: one takes a start index, and returns a 
suffix (easily confused with prefix) and the other takes a start and end 
index (the returned substring includes the character at start but not the 
character at end). 

• The key methods in StringBuilder are appendO, charAtO, delete(), 
deleteCharAtO, insert(), replace() and toStringO. 

7.1 Interconvert strings and integers 

A string is a sequence of characters. A string may encode an integer, e.g., "123" 
encodes 123. In this problem, you are to implement methods that take a string 
representing an integer and return the corresponding integer, and vice versa. Your 
code should handle negative integers. You cannot use library functions like stoi in 
C++ and parselnt in Java. 

Implement string/integer inter-conversion functions. 

Hint: Build the result one digit at a time. 

Solution: Let's consider the integer to string problem first. If the number to convert 
is a single digit, i.e., it is between 0 and 9, the result is easy to compute: it is the string 
consisting of the single character encoding that digit. 

If the number has more than one digit, it is natural to perform the conversion 
digit-by-digit. The key insight is that for any positive integer x, the least significant 
digit in the decimal representation of x is x mod 10, and the remaining digits are 
x/10. This approach computes the digits in reverse order, e.g., if we begin with 423, 
we get 3 and are left with 42 to convert. Then we get 2, and are left with 4 to convert. 
Finally, we get 4 and there are no digits to convert. The natural algorithm would be 
to prepend digits to the partial result. However, adding a digit to the beginning of a 
string is expensive, since all remaining digit have to be moved. A more time efficient 
approach is to add each computed digit to the end, and then reverse the computed 
sequence. 
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If x is negative, we record that, negate x, and then add a before reversing. If x is 
0, our code breaks out of the iteration without writing any digits, in which case we 
need to explicitly set a 0. 

To convert from a string to an integer we recall the basic working of a positional 
number system. A base-10 number d 2 d\d Q encodes the number 10 2 X d 2 +10 1 X d\ + d 0 . 
A brute-force algorithm then is to begin with the rightmost digit, and iteratively add 
10' X di to a cumulative sum. The efficient way to compute 10 ?+1 is to use the existing 
value 10* and multiply that by 10. 

A more elegant solution is to begin from the leftmost digit and with each succeed¬ 
ing digit, multiply the partial result by 10 and add that digit. For example, to convert 
"314" to an integer, we initial the partial result r to 0. In the first iteration, r - 3, in 
the second iteration r = 3xl0 + l =31, and in the third iteration r = 31xl0 + 4 = 314, 
which is the final result. 

Negative numbers are handled by recording the sign and negating the result. 

public static String intToString(int x) { 
boolean isNegative = false; 
if (x < ®) { 

isNegative = true; 

} 

StringBuilder s = new StringBuilder(); 
do { 

s.append((char)(’®’ + Math.abs(x % 1®))); 
x /= 1®; 

} while (x != ®) ; 
if (isNegative) { 

s.append(’-’); // Adds the negative sign back. 

} 

s .reverse () ; 
return s.toString(); 

} 

public static int stringToInt(String s) { 
int result = ®; 

for (int i = s.charAt(®) ==’-’? 1 : ®; i < s.length(); ++i) { 
final int digit = s.charAt(i) - ’®’; 
result = result * 1® + digit; 

} 

return s.charAt(®) == ? -result : result; 


7.2 Base conversion 

In the decimal number system, the position of a digit is used to signify the power 
of 10 that digit is to be multiplied with. For example, "314" denotes the number 
3 X 100 + 1x10 + 4x1. The base b number system generalizes the decimal number 
system: the string "^- 1^-2 • • • where 0 < 0 , < b, denotes in base-fr the integer 

a 0 x b° + a\ x b 1 + a 2 x b 1 + • • • + x b k ~ l . 
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Write a program that performs base conversion. The input is a string, an integer b \, and 
another integer b 2 . The string represents be an integer in base b\. The output should 
be the string representing the integer in base b 2 . Assume 2 < b\, b 2 < 16. Use "A" to 
represent 10, "B" for 11,..., and "F" for 15. (For example, if the string is "615", b\ is 7 
and b 2 is 13, then the result should be "1A7", since 6x7 2 + 1x7+5 = lx 13 2 +10 X13 + 7.) 

Hint: What base can you easily convert to and from? 

Solution: A brute-force approach might be to convert the input to a unary rep¬ 
resentation, and then group the Is as multiples of b 2/ fr 2 , \? v etc. For example, 
(102)3 = (llllllllll)i- To convert to base 4, there are two groups of 4 and with 
three Is remaining, so the result is (23) 4 . This approach is hard to implement, and has 
terrible time and space complexity. 

The insight to a good algorithm is the fact that all languages have an integer 
type, which supports arithmetical operations like multiply, add, divide, modulus, 
etc. These operations make the conversion much easier. Specifically, we can convert 
a string in base b\ to integer type using a sequence of multiply and adds. Then we 
convert that integer type to a string in base b 2 using a sequence of modulus and 
division operations. For example, for the string is "615", b\ — 7 and b 2 = 13, then the 
integer value, expressed in decimal, is 306. The least significant digit of the result is 
306 mod 13 = 7, and we continue with 306/13 = 23. The next digit is 23 mod 13 = 10, 
which we denote by 'A'. We continue with 23/13 = 1. Since 1 mod 13 = 1 and 
1/13 = 0, the final digit is 1, and the overall result is "1A7". Since the conversion 
alorithm is naturally expressed in terms of smaller similar subproblems, it is natural 
to implement it using recursion. 

public static String convertBase(String numAsString, int bl, int b2) { 
boolean isNegative = numAsString.startsWith("-"); 
int numAsInt = ®; 

for (int i = (isNegative ? 1 : ®) ; i < numAsString.length(); ++i) { 
numAsInt *= bl; 

numAsInt += Character.isDigit(numAsString.charAt(i)) 

? numAsString.charAt(i) - ’©’ 

: numAsString.charAt(i) - ’A’ + 1®; 

} 

return (isNegative ? : "") 

+ (numAsInt == ® ? "©" : constructFromBase(numAsInt, b2)); 

} 


private static String constructFromBase(int numAsInt, int base) { 
return numAsInt == ® 

7 "" 


} 


: constructFromBase(numAsInt / base, base) 

+ (char)(numAsInt % base >= 1® ? ’A’ + numAsInt % base - 1® 

: ’®’ + numAsInt % base); 


The time complexity is 0(n( 1 + log fo2 fri)), where n is the length of s. The reasoning is 
as follows. First, we perform n multiply-and-adds to get x from s. Then we perform 
log b2 x multiply and adds to get the result. The value x is upper-bounded by bj, and 
lo Sb 2 ( fc i') = n 1o 8i, 2 6 1- 
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7.3 Compute the spreadsheet column encoding 


Spreadsheets often use an alphabetical encoding of the successive columns. Specif¬ 
ically, columns are identified by "A", "B", "C",..., "X", "Y", "Z", "AA", "AB", ..., 
"ZZ", "AAA", "AAB",.... 

Implement a function that converts a spreadsheet column id to the corresponding 
integer, with "A" corresponding to 1. For example, you should return 4 for "D", 27 
for "AA", 702 for "ZZ", etc. How would you test your code? 

Hint: There are 26 characters in ["A", "Z"], and each can be mapped to an integer. 

Solution: A brute-force approach could be to enumerate the column ids, stopping 
when the id equals the input. The logic for getting the successor of "Z", "AZ", etc. is 
slightly involved. The bigger issue is the time-complexity—it takes 26 6 steps to get 
to "ZZZZZZ". In general, the time complexity is (9(26"), where n is the length of the 
string. 

We can do better by taking larger jumps. Specifically, this problem is basically the 
problem of converting a string representing a base-26 number to the corresponding 
integer, except that "A" corresponds to 1 not 0. We can use the string to integer 
conversion approach given in Solution 7.1 on Page 95 

For example to convert "ZZ", we initialize result to 0. We add 26, multiply by 26, 
then add 26 again, i.e., the id is 26 2 + 26 = 702. 

Good test cases are around boundaries, e.g., "A", "B", "Y", "Z","AA","AB", "ZY", 
"ZZ", and some random strings, e.g., "M", "BZ", "CCC". 

public static int ssDecodeColID(final String col) { 
int result = Q; 

for (int i = Q; i < col.length(); i++) { 
char c = col.charAt(i); 
result = result * 26 + c - ’A’ + 1; 

1 

return result; 

} 


The time complexity is 0(n). 

Variant: Solve the same problem with "A" corresponding to 0. Variant: Implement 

a function that converts an integer to the spreadsheet column id. For example, you 
should return "D" for 4, "AA" for 27, and "ZZ" for 702. 

7.4 Replace and remove 

Consider the following two rules that are to be applied to an array of characters. 

• Replace each 'a' by two 'd's. 

• Delete each entry containing a 'b'. 

For example, applying these rules to the array ( a,c,d,b,b,c,a ) results in the array 
(id , d, c, d, c, d, d). 

Write a program which takes as input an array of characters, and removes each 'b' and 
replaces each 'a' by two 'd's. Specifically, along with the array, you are provided an 
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integer-valued size. Size denotes the number of entries of the array that the operation 
is to be applied to. You do not have to worry preserving about subsequent entries. For 
example, if the array is (a, b, a, c, S) and the size is 4, then you can return ( d , d, d, d, c). 
You can assume there is enough space in the array to hold the final result. 

Hint: Consider performing multiples passes on s. 

Solution: Library array implementations often have methods for inserting into a 
specific location (all later entries are shifted right, and the array is resized) and deleting 
from a specific location (all later entries are shifted left, and the size of the array is 
decremented). If the input array had such methods, we could apply them—however, 
the time complexity would be 0{n 2 ), where n is the array's length. The reason is that 
each insertion and deletion from the array would have 0(n) time complexity. 

This problem is trivial to solve in 0(n) time if we write result to a new array—we 
skip 'b's, replace 'a's by two 'd's, and copy over all other characters. However, this 
entails 0(n) additional space. 

If there are no 'a's, we can implement the function without allocating additional 
space with one forward iteration by skipping 'b's and copying over the other charac¬ 
ters. 

If there are no 'b's, we can implement the function without additional space as 
follows. First, we compute the final length of the resulting string, which is the length 
of the array plus the number of 'a's. We can then write the result, character by 
character, starting from the last character, working our way backwards. 

For example, suppose the array is (a, c, a, a, u ), and the specified size is 4. 
Our algorithm updates the array to {a,c,a,a,^ r d,d). (Boldface denotes characters 
that are part of the final result.) The next update is {a, c, a, d, d, d, d), followed by 
{a, c, c, d, d, d, d), and finally {d, d, c, d, d, d, d). 

We can combine these two approaches to get a complete algorithm. First, we 
delete 'b's and compute the final number of valid characters of the string, with a 
forward iteration through the string. Then we replace each 'a' by two 'd's, iterating 
backwards from the end of the resulting string. If there are more 'b's than 'a's, the 
number of valid entries will decrease, and if there are more 'a's than 'b's the number 
will increase. In the program below we return the number of valid entries in the final 
result. 

public static int replaceAndRemove(int size, char[] s) { 

// Forward iteration: remove "b"s and count the number of "a"s. 
int writeldx = ®, aCount = ®; 
for (int i = ®; i < size; ++i) { 

if (s[i] != ’b’) { 

s[writeldx++] = s[i]; 

} 

if (s[i] == ’a’) { 

++aCount; 

} 

} 

// Backward iteration: replace "a"s with "dd”s starting from the end. 
int curldx = writeldx - 1; 
writeldx = writeldx + aCount - 1; 
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final int finalSize = writeldx + 1; 
while (curldx >= ®) { 
if (s[curldx] == ’a’) { 
s[writeldx--] = ’d’; 
s[writeldx--] = ’d’; 

} else { 

s[writeldx--] = s[curldx]; 

} 

--curldx; 

} 

return finalSize; 


The forward and backward iterations each take 0(n) time, so the total time complexity 
is 0(n). No additional space is allocated. 

Variant: You have an array C of characters. The characters may be letters, digits, 
blanks, and punctuation. The telex-encoding of the array C is an array T of characters 
in which letters, digits, and blanks appear as before, but punctuation marks are 
spelled out. For example, telex-encoding entails replacing the character "." by the 
string "DOT", the character "," by "COMMA", the character "?" by "QUESTION 
MARK", and the character "!" by "EXCLAMATION MARK". Design an algorithm 
to perform telex-encoding with 0(1) space. 

Variant: Write a program which merges two sorted arrays of integers, A and B. 
Specifically, the final result should be a sorted array of length m + n, where n and 
m are the lengths of A and B, respectively. Use 0(1) additional storage—assume the 
result is stored in A, which has sufficient space. These arrays are C-style arrays, i.e., 
contiguous preallocated blocks of memory. 

7.5 Test palindromicity 

For the purpose of this problem, define a palindromic string to be a string which 
when all the nonalphanumeric are removed it reads the same front to back ignoring 
case. For example, "A man, a plan, a canal, Panama." and "Able was I, ere I saw 
Elba!" are palindromic, but "Ray a Ray" is not. 

Implement a function which takes as input a string s and returns true if s is a palin¬ 
dromic string. 

Hint: Use two indices. 

Solution: The naive approach is to create a reversed version of s, and compare it with 
s, skipping nonalphanumeric characters. This requires additional space proportional 
to the length of s. 

We do not need to create the reverse—rather, we can get the effect of the reverse of s 
by traversing s from right to left. Specifically, we use two indices to traverse the string, 
one forwards, the other backwards, skipping nonalphanumeric characters, perform¬ 
ing case-insensitive comparison on the alphanumeric characters. We return false as 
soon as there is a mismatch. If the indices cross, we have verified palindromicity. 
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public static boolean isPalindrome(String s) { 

// i moves forward, and j moves backward. 
int i = ®, j = s.lengthO - 1; 
while (i < j) { 

// i and j both skip non-alphanumeric characters. 
while (!Character.isLetterOrDigit(s.charAt(i)) && i < j) { 
++i ; 

} 

while (!Character.isLetterOrDigit(s.charAt(j)) && i < j) { 

--j; 

} 

if (Character.toLowerCase(s.charAt(i++)) 

!= Character.toLowerCase(s.charAt(j--))) { 

return false; 

} 

} 

return true; 


We spend 0(1) per character, so the time complexity is 0(n), where n is the length of 
s. 


7.6 Reverse all the words in a sentence 

Given a string containing a set of words separated by whitespace, we would like to 
transform it to a string in which the words appear in the reverse order. For example, 
"Alice likes Bob" transforms to "Bob likes Alice". We do not need to keep the original 
string. 

Implement a function for reversing the words in a string s. 

Hint: It's difficult to solve this with one pass. 

Solution: The code for computing the position for each character in the final result 
in a single pass is intricate. 

However, for the special case where each word is a single character, the desired 
result is simply the reverse of s. 

For the general case, reversing s gets the words to their correct relative positions. 
However, for words that are longer than one character, their letters appear in reverse 
order. This situation can be corrected by reversing the individual words. 

For example, "ram is costly" reversed yields "yltsoc si mar". We obtain the final 
result by reversing each word to obtain "costly is ram". 

public static void reverseWords(char[] input) { 

// Reverses the whole string first. 
reverse(input, ®, input.length); 

int start = ®, end; 

while ((end = find(input, ’ start)) != -1) { 

// Reverses each word in the string. 
reverse(input, start, end); 
start = end + 1; 

} 
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// Reverses the last word. 

reverse(input, start, input.length); 

} 

public static void reverse(char[] array, int start, int stoplndex) { 
if (start >= stoplndex) { 
return; 

} 

int last = stoplndex - 1; 

for (int i = start; i <= start + (last - start) / 2; i++) { 
char tmp = array[i]; 
array[i] = array[last - i + start]; 
array[last - i + start] = tmp; 

} 

} 

public static int find(char[] array, char c, int start) { 
for (int i = start; i < array.length; i++) { 
if (array[i] == c) { 
return i; 

} 

} 

return -1; 


Since we spend 0(1) per character, the time complexity is 0(n), where n is the length 
of s. If strings are mutable, we can perform the computation in place, i.e., the 
additional space complexity is 0(1). If the string cannot be changed, the additional 
space complexity is 0(n), since we need to create a new string of length n. 


7.7 Compute all mnemonics for a phone number 

Each digit, apart from 0 and 1, in a phone keypad corresponds to one of three or four 
letters of the alphabet, as shown in Figure 7.1 on the next page. Since words are easier 
to remember than numbers, it is natural to ask if a 7 or 10-digit phone number can 
be represented by a word. For example, "2276696" corresponds to "ACRONYM" as 
well as "ABPOMZN". 

Write a program which takes as input a phone number, specified as a string of digits, 
and returns all possible character sequences that correspond to the phone number. 
The cell phone keypad is specified by a mapping that takes a digit and returns the 
corresponding set of characters. The character sequences do not have to be legal 
words or phrases. 

Hint: Use recursion. 

Solution: For a 7 digit phone number, the brute-force approach is to form 7 ranges of 
characters, one for each digit. For example, if the number is "2276696" then the ranges 
are 'A'-'C', 'A'-'C', 'P'-'S', 'M'-'O', 'M'-'O', 'W'-'Z', and 'M'-'O'. We use 7 nested 
for-loops where the iteration variables correspond to the 7 ranges to enumerate all 
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Figure 7.1: Phone keypad. 


possible mnemonics. The drawbacks of such an approach are its repetitiveness in 
code and its inflexibility. 

As a general rule, any such enumeration is best computed using recursion. The 
execution path is very similar to that of the brute-force approach, but the compiler 
handles the looping. 


public static List<String> phoneMnemonic(String phoneNumber) { 
char[] partialMnemonic = new char [phoneNumber.length()]; 

List<String> mnemonics = new ArrayList<>(); 

phoneMnemonicHelper(phoneNumber, ®, partialMnemonic, mnemonics); 
return mnemonics; 

} 

// The mapping from digit to corresponding characters. 
private static final String[] MAPPING 

= {"®" , "1", "ABC”, "DEF", "GHI", "JKL", "MNO", "PQRS", "TUV", "WXYZ"}; 

private static void phoneMnemonicHelper(String phoneNumber, int digit, 

char[] partialMnemonic, 

List<String> mnemonics) { 

if (digit == phoneNumber.length()) { 

// All digits are processed, so add partialMnemonic to mnemonics. 

// (We add a copy since subsequent calls modify partialMnemonic.) 
mnemonics. add(new String(partialMnemonic)); 

} else { 

// Try all possible characters for this digit. 

for (int i = ®; i < MAPPING[phoneNumber.charAt(digit) - ’®’].length(); 

++i) { 

char c = MAPPING[phoneNumber.charAt(digit) - ’®'].charAt(i); 
partialMnemonic[digit] = c; 

phoneMnemonicHelper(phoneNumber, digit + 1, partialMnemonic, mnemonics); 

} 

} 

} 


Since there are no more than 4 possible characters for each digit, the number of 
recursive calls, T(n), satisfies T(n ) < 4 T(n - 1), where n is the number of digits in the 
number. This solves to T(n) = (9(4”). For the function calls that entail recursion, the 
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time spent within the function, not including the recursive calls, is 0(1). Each base 
case entails making a copy of a string and adding it to the result. Since each such 
string has length n, each base case takes time 0(n). Therefore, the time complexity is 
<9(4"n). 

Variant: Solve the same problem without using recursion. 


7.8 The look-and-say problem 

The look-and-say sequence starts with 1. Subsequent numbers are derived by de¬ 
scribing the previous number in terms of consecutive digits. Specifically, to generate 
an entry of the sequence from the previous entry, read off the digits of the previ¬ 
ous entry, counting the number of digits in groups of the same digit. For exam¬ 
ple, 1; one 1; two Is; one 2 then one 1; one 1, then one 2, then two Is; three Is, 
then two 2s, then one 1. The first eight numbers in the look-and-say sequence are 
<1,11,21,1211,111221,312211,13112221,1113213211). 

Write a program that takes as input an integer n and returns the nth integer in the 
look-and-say sequence. Return the result as a string. 

Hint: You need to return the result as a string. 

Solution: We compute the nth number by iteratively applying this rule n - 1 times. 
Since we are counting digits, it is natural to use strings to represent the integers in 
the sequence. Specifically, going from the zth number to the (i + l)th number entails 
scanning the digits from most significant to least significant, counting the number of 
consecutive equal digits, and writing these counts. 


public static String lookAndSay(int n) { 
String s = "1"; 
for (int i = l; i < n; ++i) { 
s = nextNumber(s); 

} 

return s; 


private static String nextNumber(String s) { 

StringBuilder result = new StringBuilder(); 
for (int i = Q; i < s.lengthO; ++i) { 
int count = 1; 

while (i + 1 < s.lengthO && s.charAt(i) == s.charAt(i + 1)) { 
++i ; 

++count; 

} 

result.append(count); 
result.append(s.charAt(i)); 

} 

return result.toString() ; 


The precise time complexity is a function of the lengths of the terms, which is ex¬ 
tremely hard to analyze. Each successive number can have at most twice as many 
digits as the previous number—this happens when all digits are different. This means 
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the maximum length number has length no more than 2”. Since there are n iterations 
and the work in each iteration is proportional to the length of the number computed 
in the iteration, a simple bound on the time complexity is 0(n2 n ). 


7.9 Convert from Roman to decimal 

The Roman numeral representation of positive integers uses the symbols 
I, V, X, L, C, D,M. Each symbol represents a value, with I being 1, V being 5, X being 
10, L being 50, C being 100, D being 500, and M being 1000. 

In this problem we give simplified rules for representing numbers in this system. 
Specifically, define a string over the Roman number symbols to be a valid Roman 
number string if symbols appear in nonincreasing order, with the following exceptions 
allowed: 

• I can immediately precede V and X. 

• X can immediately precede L and C. 

• C can immediately precede D and M. 

Back-to-back exceptions are not allowed, e.g., IXC is invalid, as is CDM. 

A valid complex Roman number string represents the integer which is the sum 
of the symbols that do not correspond to exceptions; for the exceptions, add the 
difference of the larger symbol and the smaller symbol. 

For example, the strings "XXXXXIIIIIIIII", "LVIIH" and "LIX" are valid Roman 
number strings representing 59. The shortest valid complex Roman number string 
corresponding to the integer 59 is "LIX". 

Write a program which takes as input a valid Roman number string s and returns the 
integer it corresponds to. 

Hint: Start by solving the problem assuming no exception cases. 

Solution: The brute-force approach is to scan s from left to right, adding the value for 
the corresponding symbol unless the symbol subsequent to the one being considered 
has a higher value, in which case the pair is one of the six exception cases and the 
value of the pair is added. 

A slightly easier-to-code solution is to start from the right, and if the symbol after 
the current one is greater than it, we subtract the current symbol. The code below 
performs the right-to-left iteration. It does not check that when a smaller symbol 
appears to the left of a larger one that it is one of the six allowed exceptions, so it will, 
for example, return 99 for "IC". 

public static int romanToInteger(String s) { 

MapcCharacter, Integer> T = new HashMap<Character, Integer>() { 

{ 

put(’I’, 1); 
put(’V’, 5); 
put(’X’, 1®); 

put(’L’, 5®) ; 
put(’C’, 1®®); 

put(’D’, 5®®); 
put(’M’, 1®®®) ; 

} 
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int sum = T.get(s.charAt(s.length() - 1)); 
for (int i = s.length() - 2; i >= ®; --i) { 

if (T.get(s.charAt(i)) < T.get(s.charAt(i + 1))) { 
sum -= T.get(s.charAt(i)) ; 

} else { 

sum += T.get(s.charAt(i)) ; 

} 

} 

return sum; 


Each character of s is processed in 0(1) time, yielding an 0(n) overall time complexity, 
where n is the length of s. 

Variant: Write a program that takes as input a string of Roman number symbols and 
checks whether that string is valid. 

Variant: Write a program that takes as input a positive integer n and returns a shortest 
valid simple Roman number string representing n. 


7.10 Compute all valid IP addresses 

A decimal string is a string consisting of digits between 0 and 9. Internet Protocol 
(IP) addresses can be written as four decimal strings separated by periods, e.g., 
192.168.1.201. A careless programmer mangles a string representing an IP address 
in such a way that all the periods vanish. 

Write a program that determines where to add periods to a decimal string so that the 
resulting string is a valid IP address. There may be more than one valid IP address 
corresponding to a string, in which case you should print all possibilities. 

For example, if the mangled string is "19216811" then two corresponding IP ad¬ 
dresses are 192.168.1.1 and 19.216.81.1. (There are seven other possible IP addresses 
for this string.) 

Hint: Use nested loops. 

Solution: There are three periods in a valid IP address, so we can enumerate all 
possible placements of these periods, and check whether all four corresponding sub¬ 
strings are between 0 and 255. We can reduce the number of placements considered 
by spacing the periods 1 to 3 characters apart. We can also prune by stopping as soon 
as a substring is not valid. 

For example, if the string is "19216811", we could put the first period after "1", 
"19", and "192". If the first part is "1", the second part could be "9", "92", and "921". 
Of these, "921" is illegal so we do not continue with it. 

public static List<String> getValidlpAddress(String s) { 

List<String> result = new ArrayList<>(); 
for (int i = 1; i < 4 <&<& i < s.length(); ++i) { 
final String first = s.substring(®, i); 
if (isValidPart(first)) { 



for (int j = 1; i + j < s.length() && j <4; ++j) { 
final String second = s.substring(i, i + j); 
if (isValidPart(second)) { 

for (int k= 1; i+j +k< s.length() && k < 4; ++k) { 
final String third = s.substring(i + j, i + j + k); 
final String fourth = s.substring(i + j + k); 
if ( isValidPart (third) &<& isValidPart (fourth) ) { 

result.add(first + "." + second + + third + + fourth); 

} 

} 

} 

} 

} 

} 

return result; 


private static boolean isValidPart(String s) { 
if (s.length() > 3) { 
return false; 

} 

// "<9<9 ", ”<9<9<9 ", "Ql", etc. are not valid, but "19" is valid. 

if (s.startsWith("®") && s.length() > 1) { 
return false; 

} 

int val = Integer.parselnt(s); 
return val <= 255 && val >= ®; 


The total number of IP addresses is a constant (2 32 ), implying an 0(1) time complexity 
for the above algorithm. 

Variant: Solve the analogous problem when the number of periods is a parameter k 
and the string length is unbounded. 


7.11 Write a string sinusoidally 

We illustrate what it means to write a string in sinusoidal fashion by means 
of an example. The string "Hello^World!" written in sinusoidal fashion is 
e 1 

H 1 o W r d (Here ^ denotes a blank.) 

1 o ! 

Define the snakestring of s to be the left-right top-to-bottom sequence in which 
characters appear when s is written in sinusoidal fashion. For example, the 
snakestring string for "HelloeWorld!" is "e^lHloWrdlo!". 

Write a program which takes as input a string s and returns the snakestring of s. 

Hint: Try concrete examples, and look for periodicity. 

Solution: The brute-force approach is to populate a 3 X n 2D array of characters, 
initialized to null entries. We then write the string in sinusoidal manner in this array. 
Finally, we read out the non-null characters in row-major manner. 
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However, observe that the result begins with the characters s[l],s[5],s[9],..., fol¬ 
lowed by s[0], s[ 2], s[ 4],..., and then s[ 3], s[7], s[ 11],_Therefore, we can create the 

snakestring directly, with three iterations through s. 


public static String snakeString(String s) { 

StringBuilder result = new StringBuilder(); 

// Outputs the first row, i.e., s[l], s[5], s[9], ... 

for (int i = 1; i < s.lengthO; i += 4) { 
result.append(s.charAt(i)); 

} 

// Outputs the second row, i.e., s[9], s[2], s[4], ... 

for (int i = ®; i < s.lengthO; i += 2) { 
result.append(s.charAt(i)); 

} 

// Outputs the third row, i.e., s[3], s[7], s[ll], ... 

for (int i = 3; i < s.lengthO; i += 4) { 
result.append(s.charAt(i)); 

} 

return result.toString(); 


Let n be the length of s. Each of the three iterations takes 0(n) time, implying an 0(n) 
time complexity. 


7.12 Implement run-length encoding 

Run-length encoding (RLE) compression offers a fast way to do efficient on-the-fly 
compression and decompression of strings. The idea is simple—encode successive 
repeated characters by the repetition count and the character. For example, the RLE 
of "aaaabcccaa" is // 4alb3c2a". The decoding of "3e4f2e" returns "eeeffffee". 

Implement run-length encoding and decoding functions. Assume the string to be 
encoded consists of letters of the alphabet, with no digits, and the string to be decoded 
is a valid encoding. 

Hint: This is similar to converting between binary and string representations. 

Solution: First we consider the decoding function. Every encoded string is a rep¬ 
etition of a string of digits followed by a single character. The string of digits is 
the decimal representation of a positive integer. To generate the decoded string, we 
need to convert this sequence of digits into its integer equivalent and then write the 
character that many times. We do this for each character. 

The encoding function requires an integer (the repetition count) to string conver¬ 
sion. 


public static String decoding(String s) { 
int count = ®; 

StringBuilder result = new StringBuilder(); 
for (int i = ®; i < s.lengthO; i++) { 
char c = s.charAt(i); 
if (Character.isDigit(c)) { 
count = count * 1® + c - ’®’ ; 

} else {// c is a letter of alphabet. 
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while (count > ®) { // Appends count copies of c to result. 
result.append(c); 
count -- ; 

} 

} 

} 

return result.toString() ; 

} 

public static String encoding(String s) { 
int count = 1; 

StringBuilder ss = new StringBuilder(); 
for (int i = 1; i <= s.length(); ++i) { 

if (i == s.lengthQ || s.charAt(i) != s.charAt(i - 1)) { 

// Found new character so write the count of previous character. 

ss.append(count); 

ss.append(s.charAt(i - 1)); 

count = 1; 

} else { // s.charAt(i) == s.charAt(i - 1). 

++count; 

} 

} 

return ss.toString() ; 


The time complexity is 0{ri) r where n is the length of the string. 


7.13 Find the first occurrence of a substring 

A good string search algorithm is fundamental to the performance of many applica¬ 
tions. Several clever algorithms have been proposed for string search, each with its 
own trade-offs. As a result, there is no single perfect answer. If someone asks you this 
question in an interview, the best way to approach this problem would be to work 
through one good algorithm in detail and discuss at a high level other algorithms. 

Given two strings s (the "search string") and t (the "text"), find the first occurrence of 
s in t . 

Hint: Form a signature from a string. 

Solution: The brute-force algorithm uses two nested loops, the first iterates through t, 
the second tests if s occurs starting at the current index in t . The worst-case complexity 
is high. If t consists of n 'a's and s is w/2 'a's followed by a 'b', it will perform n/2 
unsuccessful string compares, each of which entails n /2 + 1 character compares, so 
the brute-force algorithm's time complexity is 0(n 2 ). 

Intuitively, the brute-force algorithm is slow because it advances through t one 
character at a time, and potentially does 0(m) computation with each advance, where 
m is the length of s. 

There are three linear time string matching algorithms: KMP, Boyer-Moore, and 
Rabin-Karp. Of these, Rabin-Karp is by far the simplest to understand and implement. 

The Rabin-Karp algorithm is very similar to the brute-force algorithm, but it does 
not require the second loop. Instead it uses the concept of a "fingerprint". Specifically, 
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let m be the length of s. It computes hash codes of each substring whose length is 
m —these are the fingerprints. The key to efficiency is using an incremental hash 
function, such as a function with the property that the hash code of a string is an 
additive function of each individual character. (Such a hash function is sometimes 
referred to as a rolling hash.) For such a function, getting the hash code of a sliding 
window of characters is very fast for each shift. 

For example, let the strings consist of letters from {A, C, G, T}. Suppose t is 
"GACGCCA" and s is "CGC". Define the code for "A" to be 0, the code for "C" 
to be 1, etc. Let the hash function be the decimal number formed by the integer codes 
for the letters mod 31. The hash code of s is 121 mod 31 = 28. The hash code of the 
first three characters of t, "GAC", is 201 mod 31 = 15, so s cannot be the first three 
characters of t. Continuing, the next substring of t is "ACG", whose hash code can 
be computed from 15 by subtracting 200, then multiplying by 10, then adding 2 and 
finally taking mod 31. This yields 12, so there no match yet. We then reach "CGC" 
whose hash code, 28, is derived in a similar manner. We are not done yet—there may 
be a collision. We check explicitly if the substring matches s, which in this case it 
does. 

For the Rabin-Karp algorithm to run in linear time, we need a good hash function, 
to reduce the likelihood of collisions, which entail potentially time consuming string 
equality checks. 


// Returns the index of the first character of the substring if found, -1 
// otherwise. 

public static int rabinKarp(String t, String s) { 
if (s.lengthC) > t.lengthO) { 

return -1; // s is not a substring of t. 

} 

final int BASE = 26; 

int tHash = 8, sHash = 8; // Hash codes for the substring of t and s. 
int powerS = 1; // BASE A lsf. 
for (int i = 8; i < s.length(); i++) { 
powerS = i > 8 ? powerS * BASE : 1; 
tHash = tHash * BASE + t.charAt(i); 
sHash = sHash * BASE + s.charAt(i); 

} 

for (int i = s.length(); i < t.lengthO; i++) { 

// Checks the two substrings are actually equal or not, to protect 
// against hash collision. 

if (tHash == sHash && t.substring(i - s.length(), i).equals(s)) { 
return i - s.length(); // Found a match. 

} 

// Uses rolling hash to compute the new hash code. 
tHash -= t.charAt(i - s.length()) * powerS; 
tHash = tHash * BASE + t.charAt(i); 

} 

// Tries to match s and t.substring(t.length() - s.lengthC)). 
if (tHash == sHash && t.substring(t.length() - s.length()).equals(s)) { 
return t.lengthO - s.length(); 

} 
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return -1; // s is not a substring of t. 


} 


For a good hash function, the time complexity is 0(m + n), independent of the inputs 
s and t, where m is the length of s and n is the length of t. 


Ill 



Chapter 

l Linked 


The S-expressions are formed according to the following recur¬ 
sive rules. 

1. The atomic symbols pi,pi, etc., are S-expressions. 

2. A null expression A is also admitted. 

3. If e is an S-expression so is (e). 

4. If e\ and ei are S-expressions so is {e\, ef). 

— "Recursive Functions Of Symbolic Expressions," 
J. McCarthy, 1959 


A singly linked list is a data structure that contains a sequence of nodes such that each 
node contains an object and a reference to the next node in the list. The first node 
is referred to as the head and the last node is referred to as the tail; the tail's next 
field is null. The structure of a singly linked list is given in Figure 8.1. There are 
many variants of linked lists, e.g., in a doubly linked list, each node has a link to its 
predecessor; similarly, a sentinel node or a self-loop can be used instead of null. The 
structure of a doubly linked list is given in Figure 8.2. 


□-HI UB — H ^ H —HUE 

9x1354 9x129® 9x229® 


HZE 

9x211® 


Figure 8.1: Example of a singly linked list. The number in hex below a node indicates the memory 
address of that node. 


□- hxi 2 1 - e — . 1 3 1 ~ t i — . 1 5 1 —HT F 

Figure 8.2: Example of a doubly linked list. 


Tlx] 


For all problems in this chapter, unless otherwise stated, each node has two 
entries—a data field, and a next field, which points to the next node in the list, 
with the next field of the last node being null. Its prototype is as follows: 

class ListNode<T> { 
public T data; 
public ListNode<T> next; 

} 
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Linked lists boot camp 

There are two types of list-related problems—those where you have to implement 
your own list, and those where you have to exploit the standard list library We will 
review both these aspects here, starting with implementation, then moving on to list 
libraries. 

Implementing a basic list API—search, insert, delete—for singly linked lists is an 
excellent way to become comfortable with lists. 

Search for a key: 

public static ListNode<Integer> search(ListNode<Integer> L, int key) { 
while (L ! = null <&<& L.data != key) { 

L = L.next; 

} 

// If key was not present in the list, L will have become null. 
return L; 

} 


Insert a new node after a specified node: 


// Insert newNode after node. 

public static void insertAfter(ListNode<Integer> node, 

ListNode<Integer> newNode) { 

newNode.next = node.next; 
node.next = newNode; 


Delete a node: 


// Delete the node immediately following aNode. Assumes aNode is not a tail. 
public static void deleteList(ListNode<Integer> aNode) { 
aNode.next = aNode.next.next; 

} 


Insert and delete are local operations and have 0( 1) time complexity. Search requires 
traversing the entire list, e.g., if the key is at the last node or is absent, so its time 
complexity is 0(n), where n is the number of nodes. 


List problems often have a simple brute-force solution that uses 0(n) space, but 
a subtler solution that uses the existing list nodes to reduce space complexity to 
0(1). [Problems 8.1 and 8.10] 

Very often, a problem on lists is conceptually simple, and is more about cleanly 
coding what's specified, rather than designing an algorithm. [Problem 8.12] 

Consider using a dummy head (sometimes referred to as a sentinel) to avoid 
having to check for empty lists. This simplifies code, and makes bugs less likely. 
[Problem 8.2] 

It's easy to forget to update next (and previous for double linked list) for the head 
and tail. [Problem 8.10] 

Algorithms operating on singly linked lists often benefit from using two iterators, 
one ahead of the other, or one advancing quicker than the other. [Problem 8.3] 
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Know your linked list libraries 

We now review the standard linked list library, with the reminder that many interview 
problems that are directly concerned with lists require you to write your own list class. 

Ordered sequences, including dynamically resized arrays and doubly linked 
lists, implement the List interface. The key methods for the List interface 
are add(’A’), add(2,3.14), addAll(C), addAll(Q,C), clear(), contains(2.71), 
get(12), indexOf(289), isEmptyO, iteratorO, listlteratorO, remove(l), 
removeAll(C), retainAll(C), set(3,42), subList(1, 5), and toArrayO. Be very 
comfortable with iterators—you should be able to code a class that implements the 
Iterator interface if asked to do so. 

Here are some key things to keep in mind when operating on Java lists. 

• The two most commonly used implementations of the List interface are 
ArrayList (implemented as a dynamically resized array) and LinkedList (im¬ 
plemented as a doubly linked list). Some operations are much slower on 
ArrayList, e.g., add(Q ,x) takes 0(n) time on an ArrayList, but 0(1) time on a 
LinkedList. This is because of the intrinsic difference between arrays and lists. 

• Both add(e) and remove(idx) are optional—in particular, they are not sup¬ 
ported by the object returned by Arrays. asList (), as discussed on Page 61. 

The Java Collections class consists exclusively of static methods that operate on 
or return collections. Some useful methods are Collections.addAll(C, 1,2,4), 
Collections.binarySearch(list,42), Collections.fill(list,’f’). 

Collections.swap(C,0,1), Collections.min(C), Collections.min(C,cmp), 
Collections.max(C), Collections.max(C,cmp). Collections.reverse(list). 
Collections, rotate (list, 12), Collections, sort (list), and Collections, sort (list, 
cmp). These are, by-and-large, simple functions, but will save you time, and result in 
cleaner code. 

The Arrays utility class, described on on Page 61 has a static method 
Arrays. asList () which can be very helpful in an interview. 

By way of introduction. Arrays.asList() can be applied to scalar values, e.g.. 
Arrays. asList (1,2,4) returns a list of length 3, consisting of 1, 2, and 4. It can also 
be applied to an array—it will create an object that implements the List interface, in 
0(1) time. 

Arrays.asListO can be very helpful in an interview environment, where you 
may not have the time to code up a tuple class or a class that's just a collection of 
getter/setter methods. Specifically, using Arrays. asList() you can efficiently create 
tuples, and you do not need to implement equalsQ, hashCodeO, and compareTo(): 
they will work out of the box. For example, it takes about 25 lines of code to implement 
a class implementing basic 2D-Cartesian coordinates, which you can do away with 
using Arrays.asList(xl ,yl). 

Here are some more points about Arrays. asList (). 

• The object returned by Arrays. asList (array), is partially mutable: you 
can change existing entries, but you cannot add or delete entries. 

For example, add(3.14) throws UnsupportedOperationException, but 
Collections, sort (Arrays. asList (array)), which potentially changes the 
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order of the elements, is fine. This happens because Arrays. asList (array) 
returns an adapter around the original array. 

• If you call Arrays. asList() on an array of primitive type, e.g.. 
Arrays. asList (new int [] 1,2,4), you will get a list with a single entry, which 
is likely not what you want. (That entry will be the integer array [1,2,4].) The 
box-type, i.e.. Arrays. asList (new Integer [] 1,2,4), will yield the desired re¬ 
sult. 

• To preserve the original array, make a copy of it, e.g., with Arrays. copyOf (A, 
A.length). 

8.1 Merge two sorted lists 

Consider two singly linked lists in which each node holds a number. Assume the lists 
are sorted, i.e., numbers in the lists appear in ascending order within each list. The 
merge of the two lists is a list consisting of the nodes of the two lists in which numbers 
appear in ascending order. Merge is illustrated in Figure 8.3. 


ED - 


3—ma—i 


X 


\L2Y 


\ 3 IH - H n 1X1 


9x243® ®x27®8 

(a) Two sorted lists. 


m—*TZB—HIZI3—HUES—K 


9x243® 9x124® 

(b) The merge of the lists in (a). 




Figure 8.3: Merging sorted lists. 


Write a program that takes two lists, assumed to be sorted, and returns their merge. 
The only field your program can change in a node is its next field. 

Hint: Two sorted arrays can be merged using two indices. For lists, take care when one iterator 
reaches the end. 

Solution: A naive approach is to append the two lists together and sort the resulting 
list. The drawback of this approach is that it does not use the fact that the initial 
lists are sorted. The time complexity is that of sorting, which is 0((n + m) 1 og(n + m)), 
where n and m are the lengths of each of the two input lists. 

A better approach, in terms of time complexity, is to traverse the two lists, always 
choosing the node containing the smaller key to continue traversing from. 

public static ListNode<Integer> mergeTwoSortedLists(ListNode<Integer> LI, 

ListNode<Integer> L2) { 

// Creates a placeholder for the result. 

ListNode<Integer> dummyHead = new ListNode<>(®, null); 

ListNode<Integer> current = dummyHead; 

ListNode<Integer> pi = LI, p2 = L2; 
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while (pi != null && p2 != null) { 
if (pi.data <= p2.data) { 
current.next = pi; 
pi = pi.next; 

} else { 

current.next = p2 ; 
p2 = p2.next; 

} 

current = current.next; 

} 

// Appends the remaining nodes of pi or p2. 
current.next = pi != null ? pi : p2; 
return dummyHead.next; 


The worst-case, from a runtime perspective, corresponds to the case when the lists 
are of comparable length, so the time complexity is 0(n + m). (In the best-case, one 
list is much shorter than the other and all its entries appear at the beginning of the 
merged list.) Since we reuse the existing nodes, the space complexity is (9(1). 

Variant: Solve the same problem when the lists are doubly linked. 


8.2 Reverse a single sublist 

This problem is concerned with reversing a sublist within a list. See Figure 8.4 for an 
example of sublist reversal. 

0--DD3--CZB--CZB--CZH—-CZE1 

8x279® ®x243® ®xl24® 9x183® ®xl®9® 

Figure 8.4: The result of reversing the sublist consisting of the second to the fourth nodes, inclusive, in 
the list in Figure 8.5 on the next page. 


Write a program which takes a singly linked list L and two integers s and / as 
arguments, and reverses the order of the nodes from the sth node to /th node, 
inclusive. The numbering begins at 1, i.e., the head node is the first node. Do not 
allocate additional nodes. 

Hint: Focus on the successor fields which have to be updated. 

Solution: The direct approach is to extract the sublist, reverse it, and splice it back in. 
The drawback for this approach is that it requires two passes over the sublist. 

The update can be performed with a single pass by combining the identification of 
the sublist with its reversal. We identify the start of sublist by using an iteration to get 
the sth node and its predecessor. Once we reach the sth node, we start the reversing 
process and keep counting. When we reach the /th node, we stop the reversion 
process and link the reverted section with the unreverted sections. 

public static ListNode<Integer> reverseSublist(ListNodednteger> L, int start, 

int finish) { 
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if (start == finish) { // No need to reverse since start == finish. 

return L; 

} 

ListNodednteger> dummyHead = new ListNode<>(®, L); 

ListNodednteger> sublistHead = dummyHead; 
int k = 1 ; 

while (k++ < start) { 

sublistHead = sublistHead.next; 

} 

// Reverse sublist. 

ListNode<Integer> sublistlter = sublistHead.next; 
while (start++ < finish) { 

ListNode<Integer> temp = sublistlter.next; 
sublistlter.next = temp.next; 
temp.next = sublistHead.next; 
sublistHead.next = temp; 

} 

return dummyHead.next; 


The time complexity is dominated by the search for the /th node, i.e., 0(f). 

Variant: Write a function that reverses a singly linked list. The function should use 
no more than constant storage beyond that needed for the list itself. The desired 
transformation is illustrated in Figure 8.5. 

Variant: Write a program which takes as input a singly linked list L and a nonnegative 
integer k, and reverses the list k nodes at a time. If the number of nodes n in the list is 
not a multiple of k, leave the last n mod k nodes unchanged. Do not change the data 
stored within a node. 

0-HUi 

0x270® 8x183® 0x124® 0x243® 0x100© 

Figure 8.5: The reversed list for the list in Figure 8.3(b) on Page 115. Note that no new nodes have 
been allocated. 


rzs— dir—tzb— nnu 


8.3 Test for cyclicity 

Although a linked list is supposed to be a sequence of nodes ending in null, it is 
possible to create a cycle in a linked list by making the next field of an element 
reference to one of the earlier nodes. 

Write a program that takes the head of a singly linked list and returns null if there 
does not exist a cycle, and the node at the start of the cycle, if a cycle is present. (You 
do not know the length of the list in advance.) 

Hint: Consider using two iterators, one fast and one slow. 

Solution: This problem has several solutions. If space is not an issue, the simplest 
approach is to explore nodes via the next field starting from the head and storing 
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visited nodes in a hash table—a cycle exists if and only if we visit a node already 
in the hash table. If no cycle exists, the search ends at the tail (often represented by 
having the next field set to null). This solution requires 0(n) space, where n is the 
number of nodes in the list. 

A brute-force approach that does not use additional storage and does not modify 
the list is to traverse the list in two loops—the outer loop traverses the nodes one-by- 
one, and the inner loop starts from the head, and traverses as many nodes as the outer 
loop has gone through so far. If the node being visited by the outer loop is visited 
twice, a loop has been detected. (If the outer loop encounters the end of the list, no 
cycle exists.) This approach has 0(n 2 ) time complexity. 

This idea can be made to work in linear time—use a slow iterator and a fast iterator 
to traverse the list. In each iteration, advance the slow iterator by one and the fast 
iterator by two. The list has a cycle if and only if the two iterators meet. The reasoning 
is as follows: if the fast iterator jumps over the slow iterator, the slow iterator will 
equal the fast iterator in the next step. 

Now, assuming that we have detected a cycle using the above method, we can 
find the start of the cycle, by first calculating the cycle length C. Once we know there 
is a cycle, and we have a node on it, it is trivial to compute the cycle length. To find 
the first node on the cycle, we use two iterators, one of which is C ahead of the other. 
We advance them in tandem, and when they meet, that node must be the first node 
on the cycle. 

The code to do this traversal is quite simple: 

public static ListNode<Integer> hasCycle(ListNode<Integer> head) { 
ListNode<Integer> fast = head, slow = head; 

while (fast != null && fast.next != null) { 
slow = slow.next; 
fast = fast.next.next; 
if (slow == fast) { 

// There is a cycle, so now let’s calculate the cycle length. 

int cycleLen = ®; 

do { 

++cycleLen; 
fast = fast.next; 

} while (slow != fast); 

// Finds the start of the cycle. 

ListNode<Integer> cycleLenAdvancedlter = head; 

// cycleLenAdvancedlter pointer advances cycleLen first. 

while (cycleLen-- > ®) { 

cycleLenAdvancedlter = cycleLenAdvancedlter.next; 

} 

ListNode<Integer> iter = head; 

// Both iterators advance in tandem. 

while (iter != cycleLenAdvancedlter) { 
iter = iter.next; 

cycleLenAdvancedlter = cycleLenAdvancedlter.next; 

} 

return iter; // iter is the start of cycle. 

} 


118 



} 

return null; // no cycle. 


Let F be the number of nodes to the start of the cycle, C the number of nodes on the 
cycle, and n the total number of nodes. Then the time complexity is 0(F) + 0(C) = 
0(n) — 0(F) for both pointers to reach the cycle, and 0(C) for them to overlap once the 
slower one enters the cycle. 

Variant: The following program purports to compute the beginning of the cycle 
without determining the length of the cycle; it has the benefit of being more succinct 
than the code listed above. Is the program correct? 

public static ListNode<Integer> hasCycle(ListNodednteger> head) { 

ListNodednteger> fast = head, slow = head; 

while (fast != null && fast.next != null && fast.next.next != null) { 
slow = slow.next; 
fast = fast.next.next; 

if (slow == fast) { // There is a cycle. 

// Tries to find the start of the cycle. 
slow = head; 

// Both pointers advance at the same time. 
while (slow != fast) { 
slow = slow.next; 
fast = fast.next; 

} 

return slow; // slow is the start of cycle. 

} 

} 

return null; // No cycle. 

} 


8.4 Test for overlapping lists—lists are cycle-free 

Given two singly linked lists there may be list nodes that are common to both. (This 
may not be a bug—it may be desirable from the perspective of reducing memory 
footprint, as in the flyweight pattern, or maintaining a canonical form.) For example, 
the lists in Figure 8.6 overlap at Node I. 



Figure 8.6: Example of overlapping lists. 


Write a program that takes two cycle-free singly linked lists, and determines if there 
exists a node that is common to both lists. 
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Hint: Solve the simple cases first. 

Solution: A brute-force approach is to store one list's nodes in a hash table, and then 
iterate through the nodes of the other, testing each for presence in the hash table. This 
takes 0(n) time and 0(n) space, where n is the total number of nodes. 

We can avoid the extra space by using two nested loops, one iterating through the 
first list, and the other to search the second for the node being processed in the first 
list. However, the time complexity is 0(n 2 ). 

The lists overlap if and only if both have the same tail node: once the lists converge 
at a node, they cannot diverge at a later node. Therefore, checking for overlap amounts 
to finding the tail nodes for each list. 

To find the first overlapping node, we first compute the length of each list. The 
first overlapping node is determined by advancing through the longer list by the 
difference in lengths, and then advancing through both lists in tandem, stopping at 
the first common node. If we reach the end of a list without finding a common node, 
the lists do not overlap. 

public static ListNode<Integer> overlappingNoCycleLists( 

ListNode<Integer> LI, ListNode<Integer> L2) { 
int LILength = length(Ll), L2Length = length(L2); 

// Advances the longer list to get equal length lists. 
if (LILength > L2Length) { 

LI = advanceListByK(LILength - L2Length, LI); 

} else { 

L2 = advanceListByK(L2Length - LILength, L2); 

} 

while (LI ! = null && L2 != null && LI != L2) { 

LI = LI.next; 

L2 = L2.next; 

} 

return LI; // nullptr implies there is no overlap between LI and L2. 


public static ListNode<Integer> advanceListByK(int k, ListNode<Integer> L) { 
while (k-- > Q) { 

L = L.next; 

} 

return L; 


private static int length(ListNode<Integer> L) { 
int len = ®; 
while (L != null) { 

++len; 

L = L.next; 

} 

return len; 


The time complexity is 0(n) and the space complexity is 0(1). 
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8.5 Test for overlapping lists—lists may have cycles 


Solve Problem 8.4 on Page 119 for the case where the lists may each or both have 
a cycle. If such a node exists, return a node that appears first when traversing the 
lists. This node may not be unique—if one node ends in a cycle, the first cycle node 
encountered when traversing it may be different from the first cycle node encountered 
when traversing the second list, even though the cycle is the same. In such cases, you 
may return either of the two nodes. 

For example. Figure 8.7 shows an example of lists which overlap and have cycles. 
For this example, both A and B are acceptable answers. 



Figure 8.7: Overlapping lists. 


Hint: Use case analysis. What if both lists have cycles? What if they end in a common cycle? 
What if one list has cycle and the other does not? 

Solution: This problem is easy to solve using 0(n) time and space complexity, where 
n is the total number of nodes, using the hash table approach in Solution 8.4 on the 
facing page. 

We can improve space complexity by studying different cases. The easiest case is 
when neither list is cyclic, which we can determine using Solution 8.3 on Page 117. 
In this case, we can check overlap using the technique in Solution 8.4 on the facing 
page. 

If one list is cyclic, and the other is not, they cannot overlap, so we are done. 

This leaves us with the case that both lists are cyclic. In this case, if they overlap, 
the cycles must be identical. 

There are two subcases: the paths to the cycle merge before the cycle, in which case 
there is a unique first node that is common, or the paths reach the cycle at different 
nodes on the cycle. For the first case, we can use the approach of Solution 8.4 on 
the preceding page. For the second case, we use the technique in Solution 8.3 on 
Page 117. 

public static ListNode<Integer> overlappingLists(ListNodednteger> LI, 

ListNodednteger> L2) { 

// Store the start of cycle if any. 

ListNode<Integer> rootl = CheckingCycle.hasCycle(LI); 

ListNode<Integer> root2 = CheckingCycle.hasCycle(L2); 

if (rootl == null <&& root2 == null) { 

// Both lists don’t have cycles. 
return overlappingNoCycleLists(LI, L2); 
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} else if ((rootl != null &<& root2 == null) 

|| (rootl == null && root2 != null)) { 

// One list has cycle, and one list has no cycle. 

return null; 

} 

// Both lists have cycles. 

ListNodednteger> temp = root2; 
do { 

temp = temp.next; 

} while (temp != rootl && temp != root2); 

// LI and L2 do not end in the same cycle. 
if (temp != rootl) { 

return null; // Cycles are disjoint. 

} 

// LI and L2 end in the same cycle, locate the overlapping node if they 
// first overlap before cycle starts. 

int stemlLength = distance(LI, rootl), stem2Length = distance(L2, root2); 
int count = Math.abs(stemlLength - stem2Length); 
if (stemlLength > stem2Length) { 

LI = advanceListByK(stemlLength - stem2Length, LI); 

} else { 

L2 = advanceListByK(stem2Length - stemlLength, L2); 

} 

while (LI != L2 && LI != rootl && L2 != root2) { 

LI = LI.next; 

L2 = L2.next; 

} 

// If LI == L2 before reaching rootl, it means the overlap first occurs 
// before the cycle starts; otherwise, the first overlapping node is not 
// unique, so we can return any node on the cycle. 
return LI == L2 ? LI : rootl; 


// Calculates the distance between a and b. 

private static int distance(ListNode<Integer> a, ListNode<Integer> b) { 
int dis = ®; 
while (a != b) { 
a = a.next; 

++dis; 

} 

return dis; 


The algorithm has time complexity 0(n + m), where n and m are the lengths of the 
input lists, and space complexity 0(1). 

8.6 Delete a node from a singly linked list 

Given a node in a singly linked list, deleting it in 0(1) time appears impossible because 
its predecessor's next field has to be updated. Surprisingly, it can be done with one 
small caveat—the node to delete cannot be the last one in the list and it is easy to copy 
the value part of a node. 
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Write a program which deletes a node in a singly linked list. The input node is 
guaranteed not to be the tail node. 

Hint: Instead of deleting the node, can you delete its successor and still achieve the desired 
configuration? 

Solution: Given the pointer to a node, it is impossible to delete it from the list without 
modifying its predecessor's next pointer and the only way to get to the predecessor 
is to traverse the list from head, which requires 0(n) time, where n is the number of 
nodes in the list. 

Given a node, it is easy to delete its successor, since this just requires updating the 
next pointer of the current node. If we copy the value part of the next node to the 
current node, and then delete the next node, we have effectively deleted the current 
node. The time complexity is 0(1). 

// Assumes nodeToDelete is not tail. 

public static void deletionFromList(ListNodednteger > nodeToDelete) { 
nodeToDelete.data = nodeToDelete.next.data; 
nodeToDelete.next = nodeToDelete.next.next; 

} 


8.7 Remove the km last element from a list 

Without knowing the length of a linked list, it is not trivial to delete the kth last 
element in a singly linked list. 

Given a singly linked list and an integer k, write a program to remove the kth last 
element from the list. Your algorithm cannot use more than a few words of storage, 
regardless of the length of the list. In particular, you cannot assume that it is possible 
to record the length of the list. 

Hint: If you know the length of the list, can you find the kth last node using two iterators? 

Solution: A brute-force approach is to compute the length with one pass, and then 
use that to determine which node to delete in a second pass. A drawback of this 
approach is that it entails two passes over the data, which is slow, e.g., if traversing 
the list entails disc accesses. 

We use two iterators to traverse the list. The first iterator is advanced by k steps, 
and then the two iterators advance in tandem. When the first iterator reaches the tail, 
the second iterator is at the (k + l)th last node, and we can remove the kth node. 

// Assumes L has at least k nodes, deletes the k-th last node in L. 
public static ListNode<Integer> removeKthLast(ListNodecInteger> L, int k) { 
ListNodecInteger> dummyHead = new ListNodeo (®, L); 

ListNodecInteger> first = dummyHead.next; 
while (k-- > ®) { 
first = first.next; 

} 

ListNodecInteger> second = dummyHead; 
while (first != null) { 
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second = second.next; 
first = first.next; 

} 

// second points to the (k + l)-th last node, deletes its successor. 
second.next = second.next.next; 
return dummyHead.next; 


The time complexity is that of list traversal, i.e., 0(n), where n is the length of the list. 
The space complexity is 0(1), since there are only two iterators. 

Compared to the brute-force approach, if k is small enough that we can keep the set 
of nodes between the two iterators in memory, but the list is too big to fit in memory, 
the two-iterator approach halves the number of disc accesses. 


8.8 Remove duplicates from a sorted list 

This problem is concerned with removing duplicates from a sorted list of integers. 
See Figure 8.8 for an example. 


|T|- H 2 2 | «3 - H 3 I H - H l I H - H l I H » f n I «+ » T n |X| 


8x188® 8x211® 8x183® 8x124® 8x2288 8x128® 8x1354 

(a) List before removing duplicates. 

CZB—CZH—TZS—CZS-^QD! 

8x188® 8x183® 8x124® 8x228® 8x128® 

(b) The list in (a) after removing duplicates. 


Figure 8.8: Example of duplicate removal. 


Write a program that takes as input a singly linked list of integers in sorted order, and 
removes duplicates from it. The list should be sorted. 

Hint: Focus on the successor fields which have to be updated. 

Solution: A brute-force algorithm is to create a new list, using a hash table to test if 
a value has already been added to the new list. Alternatively, we could search in the 
new list itself to see if the candidate value already is present. If the length of the list is 
n, the first approach requires 0(n) additional space for the hash table, and the second 
requires 0(n 2 ) time to perform the lookups. Both allocate n nodes for the new list. 

A better approach is to exploit the sorted nature of the list. As we traverse the list, 
we remove all successive nodes with the same value as the current node. 

public static ListNodecInteger> removeDuplicates(ListNodednteger> L) { 
ListNodednteger> iter = L; 
while (iter != null) { 

// Uses nextDistinct to find the next distinct value. 

ListNode<Integer> nextDistinct = iter.next; 

while (nextDistinct != null <&<& nextDistinct. data == iter.data) { 
nextDistinct = nextDistinct.next; 

} 
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3 X 


0x22®® ®xl®0® ®x211® 9x1354 0x120® 

Figure 8.9: The result of applying a right cyclic shift by 3 to the list in Figure 8.1 on Page 112. Note that 
no new nodes have been allocated. 


iter.next = nextDistinct; 
iter = nextDistinct; 

} 

return L; 


Determining the time complexity requires a little amortized analysis. A single node 
may take more than 0(1) time to process if there are many successive nodes with 
the same value. A clearer justification for the time complexity is that each link is 
traversed once, so the time complexity is 0(n). The space complexity is 0(1). 

Variant: Let m be a positive integer and L a sorted singly linked list of integers. For 
each integer k, if k appears more than m times in L, remove all nodes from L containing 
k. 

8.9 Implement cyclic right shift for singly linked lists 

This problem is concerned with performing a cyclic right shift on a list. 

Write a program that takes as input a singly linked list and a nonnegative integer k, 
and returns the list cyclically shifted to the right by k. See Figure 8.9 for an example 
of a cyclic right shift. 

Hint: How does this problem differ from rotating an array? 

Solution: A brute-force strategy is to right shift the list by one node k times. Each 
right shift by a single node entails finding the tail, and its predecessor. The tail is 
prepended to the current head, and its original predecessor's successor is set to null 
to make it the new tail. The time complexity is 0(kn), and the space complexity is 
(9(1), where n is the number of nodes in the list. 

Note that k may be larger than n. If so, it is equivalent to shift by k mod n, so 
we assume k < n. The key to improving upon the brute-force approach is to use the 
fact that linked lists can be cut and the sublists reassembled very efficiently. First we 
find the tail node t. Since the successor of the tail is the original head, we update t's 
successor. The original head is to become the kth node from the start of the new list. 
Therefore, the new head is the (n - k )th node in the initial list. 

public static ListNode<Integer> cyclicallyRightShiftList(ListNodednteger> L, 

int k) { 

if (L == null) { 
return L; 

} 

// Computes the length of L and the tail. 
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ListNodecInteger> tail = L; 
int n = 1; 

while (tail.next != null) { 

++n; 

tail = tail.next; 

} 

k %= n; 
if (k == ®) { 

return L; 

} 

tail.next = L; // Makes a cycle by connecting the tail to the head. 
int stepsToNewHead = n - k; 

ListNodecInteger> newTail = tail; 
while (stepsToNewHead-- > ®) { 
newTail = newTail.next; 

} 

ListNodecInteger> newHead = newTail.next; 
newTail.next = null; 
return newHead; 


The time complexity is 0(n), and the space complexity is (9(1). 


8.10 Implement even-odd merge 

Consider a singly linked list whose nodes are numbered starting at 0. Define the even- 
odd merge of the list to be the list consisting of the even-numbered nodes followed 
by the odd-numbered nodes. The even-odd merge is illustrated in Figure 8.10. 


0 


0 


-+un3-^nn3-^nn3-^un3-^±M 

8x188® <9x124® 8x183® 8x211® 8x228® 

(a) The initial list is L. The number in hex below a node indicates the memory address of that node. 
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(b) The even-odd merge of L—note that no new nodes have been allocated. 


Figure 8.10: Even-odd merge example. 


Write a program that computes the even-odd merge. 

Hint: Use temporary additional storage. 

Solution: The brute-force algorithm is to allocate new nodes and compute two new 
lists, one for the even and one for the odd nodes. The result is the first list concatenated 
with the second list. The time and space complexity are both 0(n). 

However, we can avoid the extra space by reusing the existing list nodes. We do 
this by iterating through the list, and appending even elements to one list and odd 
elements to another list. We use an indicator variable to tell us which list to append 
to. Finally we append the odd list to the even list. 
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public static ListNodecInteger> evenOddMerge(ListNodecInteger> L) { 

if (L == null) { 
return L; 

} 

ListNodednteger > evenDummyHead = new ListNode<>(®, null), 
oddDummyHead = new ListNode<>(® , null); 

ListcListNodednteger >> tails = Arrays . asList ( evenDummyHead , oddDummyHead); 

int turn = ®; 

for (ListNodednteger> iter = L; iter != null; iter = iter.next) { 
tails.get(turn).next = iter; 
tails.set(turn, tails.get(turn).next); 
turn A = 1; 

} 

tails.get( 1) .next = null; 

tails.get(®).next = oddDummyHead.next; 

return evenDummyHead.next; 


The time complexity is 0(n) and the space complexity is 0(1). 


8.11 Test whether a singly linked list is palindromic 

It is straightforward to check whether the sequence stored in an array is a palindrome. 
However, if this sequence is stored as a singly linked list, the problem of detecting 
palindromicity becomes more challenging. See Figure 8.1 on Page 112 for an example 
of a palindromic singly linked list. 

Write a program that tests whether a singly linked list is palindromic. 

Hint: It's easy if you can traverse the list forwards and backwards simultaneously. 

Solution: A brute-force algorithm is to compare the first and last nodes, then the 
second and second-to-last nodes, etc. The time complexity is 0(n 2 ), where n is the 
number of nodes in the list. The space complexity is 0(1). 

The 0(n 2 ) complexity comes from having to repeatedly traverse the list to identify 
the last, second-to-last, etc. Getting the first node in a singly linked list is an 0(1) time 
operation. This suggests paying a one-time cost of 0(n) time complexity to get the 
reverse of the second half of the original list, after which testing palindromicity of the 
original list reduces to testing if the first half and the reversed second half are equal. 
This approach changes the list passed in, but the reversed sublist can be reversed 
again to restore the original list. 

public static boolean isLinkedListAPalindrome(ListNode<Integer> L) { 
if (L == null) { 
return true; 

} 

// Finds the second half of L. 

ListNodecInteger> slow = L, fast = L; 
while (fast != null && fast.next != null) { 
fast = fast . next. next; 
slow = slow.next; 
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} 


// Compare the first half and the reversed second half lists. 
ListNodecInteger> firstHalfIter = L; 

ListNodecInteger> secondHalfIter 

= ReverseLinkedListlterative.reverseLinkedList(slow); 
while (secondHalfIter != null && firstHalfIter != null) { 
if (secondHalflter.data != firstHalfIter.data) { 
return false; 

} 

secondHalflter = secondHalfIter.next; 
firstHalfIter = firstHalfIter.next; 

} 

return true; 


The time complexity is O(n). The space complexity is 0(1). 

Variant: Solve the same problem when the list is doubly linked and you have pointers 
to the head and the tail. 


8.12 Implement list pivoting 

For any integer k, the pivot of a list of integers with respect to k is that list with its 
nodes reordered so that all nodes containing keys less than k appear before nodes 
containing k, and all nodes containing keys greater than k appear after the nodes 
containing k. See Figure 8.11 for an example of pivoting. 
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(a) A list to be pivoted. 
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(b) The result of pivoting the list in (a) with k = 7. 

Figure 8.11: List pivoting. 


Implement a function which takes as input a singly linked list and an integer k and 
performs a pivot of the list with respect to k. The relative ordering of nodes that 
appear before k, and after k, must remain unchanged; the same must hold for nodes 
holding keys equal to k. 

Hint: Form the three regions independently. 

Solution: A brute-force approach is to form three lists by iterating through the list 
and writing values into one of the three new lists based on whether the current value 
is less than, equal to, or greater than k. We then traverse the list from the head, and 
overwrite values in the original list from the less than, then equal to, and finally 
greater than lists. The time and space complexity are 0(n), where n is the number of 
nodes in the list. 
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A key observation is that we do not really need to create new nodes for the three 
lists. Instead we reorganize the original list nodes into these three lists in a single 
traversal of the original list. Since the traversal is in order, the individual lists preserve 
the ordering. We combine these three lists in the final step. 

public static ListNode<Integer> listPivoting(ListNodecInteger> L, int x) { 
ListNodecInteger> lessHead = new ListNodec>(®, null); 

ListNodecInteger> equalHead = new ListNodec>(®, null); 

ListNodecInteger> greaterHead = new ListNodec>(®, null); 

ListNodecInteger> lesslter = lessHead; 

ListNodecInteger> equallter = equalHead; 

ListNodecInteger> greaterlter = greaterHead; 

// Populates the three lists. 

ListNodecInteger> iter = L; 
while (iter != null) { 
if (iter.data c x) { 
lesslter.next = iter; 
lesslter = iter; 

} else if (iter.data == x) { 
equallter.next = iter; 
equallter = iter; 

} else { // iter.data > x. 
greaterlter.next = iter; 
greaterlter = iter; 

} 

iter = iter.next; 

} 

// Combines the three lists. 
greaterlter.next = null; 
equallter.next = greaterHead.next; 
lesslter.next = equalHead.next; 
return lessHead.next; 

} 


The time to compute the three lists is 0(n). Combining the lists takes 0(1) time, 
yielding an overall 0(n) time complexity. The space complexity is 0(1). 


8.13 Add list-based integers 

A singly linked list whose nodes contain digits can be viewed as an integer, with the 
least significant digit coming first. Such a representation can be used to represent 
unbounded integers. This problem is concerned with adding integers represented in 
this fashion. See Figure 8.12 for an example. 

0 - H 3 M - H l M - H 4 1X1 0 - H 7 H - H o IH - H 9 Ixl 

(a) Two lists. 

m —*0113—*0113—HIZB—HXIEI 

(b) The sum of the two lists in (a). 

Figure 8.12: List-based interpretation of 413 + 907 = 1320. 
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Write a program which takes two singly linked lists of digits, and returns the list 
corresponding to the sum of the integers they represent. The least significant digit 
comes first. 

Hint: First, solve the problem assuming no pair of corresponding digits sum to more than 9. 

Solution: Note that we cannot simply convert the lists to integers, since the integer 
word length is fixed by the machine architecture, and the lists can be arbitrarily long. 

Instead we mimic the grade-school algorithm, i.e., we compute the sum of the 
digits in corresponding nodes in the two lists. A key nuance of the computation is 
handling the carry-out from a particular place. Care has to be taken to remember to 
allocate an additional node if the final carry is nonzero. 

public static ListNode<Integer> addTwoNumbers(ListNode<Integer> LI, 

ListNodecInteger> L2) { 

ListNodecInteger> dummyHead = new ListNode<>(©, null); 

ListNodecInteger> placelter = dummyHead; 
int carry = ®; 

while (LI != null || L2 != null) { 
int sum = carry; 
if (LI != null) { 
sum += LI.data; 

LI = LI.next; 

} 

if (L2 != null) { 
sum += L2.data; 

L2 = L2.next; 

} 

placelter.next = new ListNode<>(sum % 1®, null); 
carry = sum / 1®; 
placelter = placelter.next; 

} 

// carry cannot exceed 1, so we never need to add more than one node. 
if (carry > ®) { 

placelter . next = new ListNodeo(carry , null); 

} 

return dummyHead.next; 

} 


The time complexity is 0(n + m) and the space complexity is 0(max(n, m)), where n 
and m are the lengths of the two lists. 

Variant: Solve the same problem when integers are represented as lists of digits with 
the most significant digit comes first. 
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Chapter 


I Stacks and Queues 

Linear lists in which insertions, deletions, and accesses 
to values occur almost always at the first or the last 
node are very frequently encountered, and we give 
them special names ... 

— "The Art of Computer Programming, Volume 1," 
D. E. Knuth, 1997 

Stacks 

A stack supports two basic operations—push and pop. Elements are added (pushed) 
and removed (popped) in last-in, first-out order, as shown in Figure 9.1. If the stack 
is empty, pop typically returns null or throws an exception. 

When the stack is implemented using a linked list these operations have 0(1) 
time complexity. If it is implemented using an array, there is maximum number of 
entries it can have—push and pop are still 0(1). If the array is dynamically resized, 
the amortized time for both push and pop is 0(1). A stack can support additional 
operations such as peek, which returns the top of the stack without popping it. 


pop push 3 



(a) Initial configuration. (b) Popping (a). (c) Pushing 3 on to (b). 

Figure 9.1 : Operations on a stack. 

Stacks boot camp 

The last-in, first-out semantics of a stack make it very useful for creating reverse 
iterators for sequences which are stored in a way that would make it difficult or 
impossible to step back from a given element. This a program uses a stack to print 
the entries of a singly-linked list in reverse order. 

public static void printLinkedListlnReverse(ListNodecInteger> head) { 
Dequednteger> nodes = new LinkedList<>() ; 
while (head != null) { 
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nodes.addFirst(head.data); 
head = head.next; 


} 

while (!nodes.isEmpty()) { 

System.out.printIn(nodes.poll()) ; 

} 


The time and space complexity are 0(n), where n is the number of nodes in the list. 

As an alternative, we could form the reverse of the list using Solution 8.2 on 
Page 116, iterate through the list printing entries, then perform another reverse to 
recover the list—this would have 0(n) time complexity and 0(1) space complexity. 


Learn to recognize when the stack LIFO property is applicable. For example, 
parsing typically benefits from a stack. [Problems 9.2 and 9.6] 

Consider augmenting the basic stack or queue data structure to support additional 
operations, such as finding the maximum element. [Problem 9.1] 


Know your stack libraries 

The preferred way to represent stacks in Java is via the Deque interface. The 
LinkedList class is a doubly linked list that implements this interface, and provides 
efficient (0(1) time) stack (and queue) functionality. 

The key stack-related methods in the Deque interface are push(42), peek(), and 
pop(). The Deque methods addFirst(42), peekFirstO, and removeFirst() are 
identical to push(42), peek(), and pop(). We use them in our solutions in place of 
push(42), peek(), and pop(). 

• push(e) pushes an element onto the stack. Not much can go wrong with a call 
to push: some implementing classes may throw an IllegalStateException if 
the capacity limit is exceeded, or a NullPointerException if the element being 
inserted is null. LinkedList has no capacity limit and allows for null entries, 
though as we will see you should be very careful when adding null. 

• peek() will retrieve, but does not remove, the element at the top of the stack. 
If the stack is empty, it returns null. Since null may be a valid entry, this leads 
to ambiguity. Therefore a better test for an empty stack is isEmpty (). 

• pop() will remove and return the element at the top of the stack. It throws 
NoSuchElementException if the deque is empty. To avoid the exception, first 
test with isEmptyO. 

Other useful stack-related methods are descendinglteratorO and iterator(). 

9.1 Implement a stack with max API 

Design a stack that includes a max operation, in addition to push and pop. The max 
method should return the maximum value stored in the stack. 

Hint: Use additional storage to track the maximum value. 

Solution: The simplest way to implement a max operation is to consider each element 
in the stack, e.g., by iterating through the underlying array for an array-based stack. 
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The time complexity is 0(n) and the space complexity is 0( 1), where n is the number 
of elements currently in the stack. 

The time complexity can be reduced to <9(log n ) using auxiliary data structures, 
specifically, a heap or a BST, and a hash table. The space complexity increases to 0(n) 
and the code is quite complex. 

Suppose we use a single auxiliary variable, M, to record the element that is max¬ 
imum in the stack. Updating M on pushes is easy: M = max(M, e ), where e is the 
element being pushed. However, updating M on pop is very time consuming. If 
M is the element being popped, we have no way of knowing what the maximum 
remaining element is, and are forced to consider all the remaining elements. 

We can dramatically improve on the time complexity of popping by caching, in 
essence, trading time for space. Specifically, for each entry in the stack, we cache the 
maximum stored at or below that entry. Now when we pop, we evict the correspond¬ 
ing cached value. 

private static class ElementWithCachedMax { 
public Integer element; 
public Integer max; 

public ElementWithCachedMax(Integer element, Integer max) { 
this.element = element; 
this .max = max; 

} 

} 

public static class Stack { 

// Stores (element, cached maximum) pair. 
private Deque<ElementWithCachedMax> elementWithCachedMax 
= new LinkedList<>(); 

public boolean empty() { return elementWithCachedMax.isEmpty(); } 

public Integer max() { 
if (empty()) { 

throw new IllegalStateException("max(): empty stack"); 

} 

return elementWithCachedMax.peek().max; 

} 

public Integer pop() { 
if (empty()) { 

throw new IllegalStateException("pop(): empty stack"); 

} 

return elementWithCachedMax.removeFirst().element; 

} 

public void push(Integer x) { 
elementWithCachedMax.addFirst( 

new ElementWithCachedMax(x, Math.max(x, empty() ? x : max()))); 

} 

} 
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Each of the specified methods has time complexity 0(1). The additional space com¬ 
plexity is 0(n), regardless of the stored keys. 

We can improve on the best-case space needed by observing that if an element e 
being pushed is smaller than the maximum element already in the stack, then e can 
never be the maximum, so we do not need to record it. We cannot store the sequence 
of maximum values in a separate stack because of the possibility of duplicates. We 
resolve this by additionally recording the number of occurrences of each maximum 
value. See Figure 9.2 on the next page for an example. 

private static class MaxWithCount { 
public Integer max; 
public Integer count; 

public MaxWithCount(Integer max, Integer count) { 
this .max = max; 
this .count = count; 

} 

} 

public static class Stack { 

private Dequednteger> element = new LinkedList<>() ; 

private Deque<MaxWithCount> cachedMaxWithCount = new LinkedList<>(); 

public boolean empty() { return element.isEmpty(); } 

public Integer max() { 
if (empty()) { 

throw new IllegalStateException("max(): empty stack"); 

} 

return cachedMaxWithCount.peekFirst().max; 

} 

public Integer pop() { 
if (empty()) { 

throw new IllegalStateException("pop(): empty stack"); 

} 

Integer popElement = element.removeFirst(); 

if (popElement.equals(cachedMaxWithCount.peekFirst().max)) { 
cachedMaxWithCount.peekFirst().count 

= cachedMaxWithCount.peekFirst().count - 1; 
if (cachedMaxWithCount.peekFirst().count.equals(®)) { 
cachedMaxWithCount.removeFirst(); 

} 

} 

return popElement; 

} 

public void push(Integer x) { 
element.addFirst (x) ; 

if (! cachedMaxWithCount.isEmpty()) { 

if (Integer.compare(x, cachedMaxWithCount.peekFirst().max) == ®) { 
cachedMaxWithCount.peekFirst().count 

= cachedMaxWithCount.peekFirst().count + 1; 

} else if (Integer.compare(x, cachedMaxWithCount.peekFirst().max) > ®) { 
cachedMaxWithCount.addFirst(new MaxWithCount(x, 1)); 
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} 

} else { 

cachedMaxWithCount.addFirst (new MaxWithCount(x, 1)); 

} 

} 

} 


The worst-case additional space complexity is <9(n), which occurs when each key 
pushed is greater than all keys in the primary stack. However, when the number 
of distinct keys is small, or the maximum changes infrequently, the additional space 
complexity is less, 0( 1) in the best-case. The time complexity for each specified 
method is still 0(1 ). 




Figure 9.2: The primary and auxiliary stacks for the following operations: push 2, push 2, push 1, 
push 4, push 5, push 5, push 3, pop, pop, pop, pop, push 0, push 3. Both stacks are initially empty, and 
their progression is shown from left-to-right, then top-to-bottom. The top of the auxiliary stack holds the 
maximum element in the stack, and the number of times that element occurs in the stack. The auxiliary 
stack is denoted by aux. 


9.2 Evaluate RPN expressions 

A string is said to be an arithmetical expression in Reverse Polish notation (RPN) if: 
(1.) It is a single digit or a sequence of digits, prefixed with an option -, e.g., "6", 
"123", "-42". 

(2.) It is of the form " A , B, o" where A and B are RPN expressions and o is one of 
+/-/X,/. 

For example, the following strings satisfy these rules: "1729", "3,4,+,2,X, 1,+", 
"1,1, +, -2, x", "-641,6, /, 28, /". 

An RPN expression can be evaluated uniquely to an integer, which is determined 
recursively. The base case corresponds to Rule (1.), which is an integer expressed in 
base-10 positional system. Rule (2.)corresponds to the recursive case, and the RPNs 
are evaluated in the natural way, e.g., if A evaluates to 2 and B evaluates to 3, then 
"A, B, x" evaluates to 6. 
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Write a program that takes an arithmetical expression in RPN and returns the number 
that the expression evaluates to. 

Hint: Process subexpressions, keeping values in a stack. How should operators be handled? 

Solution: Let's begin with the RPN example "3,4, +, 2, X, 1, + The ordinary form for 
this is (3 + 4) X 2 + 1. To evaluate this by hand, we would scan from left to right. We 
record 3, then 4, then applying the + to 3 and 4, and record the result, 7. Note that 
we never need to examine the 3 and 4 again. Next we multiply by 2, and record the 
result, 14. Finally, we add 1 to obtain the final result, 15. 

Observe that we need to record partial results, and as we encounter operators, 
we apply them to the partial results. The partial results are added and removed in 
last-in, first-out order, which makes a stack the natural data structure for evaluating 
RPN expressions. 

public static int eval(String RPNExpression) { 

Dequednteger> intermediateResults = new LinkedList<>() ; 

String delimiter = 

String[] symbols = RPNExpression.split(delimiter); 
for (String token : symbols) { 

if (token.length() == 1 && contains(token)) { 

final int y = intermediateResults.removeFirst(); 
final int x = intermediateResults.removeFirst(); 
switch (token.charAt(Q)) { 
case ’: 

intermediateResults.addFirst(x + y); 

break; 
case ’-’: 

intermediateResults.addFirst(x - y); 

break; 
case ’*’: 

intermediateResults.addFirst(x * y); 

break; 
case ’/’: 

intermediateResults.addFirst(x / y); 

break; 
default: 

throw new IllegalArgumentException("Malformed RPN at :" + token); 

} 

} else { // token is a number. 

intermediateResults.addFirst(Integer.parseInt(token)); 

} 

} 

return intermediateResults.removeFirst(); 

} 


Since we perform 0(1) computation per character of the string, the time complexity 
is 0(n), where n is the length of the string. 

Variant: Solve the same problem for expressions in Polish notation, i.e., when A, B, o 
is replaced by o. A, B in Rule (2.) on the preceding page. 
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9.3 Test a string over for well-formedness 

A string over the characters is said to be well-formed if the different types 

of brackets match in the correct order. 

For example, "([]){()}" is well-formed, as is "[()[]{()()}]". However, "{)" and 
"[()[]{()()" are not well-formed. 

Write a program that tests if a string made up of the characters '(',')','[',and"}' 
is well-formed. 

Hint: Which left parenthesis does a right parenthesis match with? 

Solution: Let's begin with well-formed strings consisting solely of left and right 
parentheses, e.g., "()(())". If such a string is well-formed, each right parenthesis must 
match the closest left parenthesis to its left. Therefore, starting from the left, every 
time we see a left parenthesis, we store it. Each time we see a right parenthesis, we 
match it with a stored left parenthesis. Since there are not brackets or braces, we can 
simply keep a count of the number of unmatched left parentheses. 

For the general case, we do the same, except that we need to explicitly store the 
unmatched left characters, i.e., left parenthesis, left brackets, and left braces. We 
cannot use three counters, because that will not tell us the last unmatched one. A 
stack is a perfect option for this application: we use it to record the unmatched left 
characters, with the most recent one at the top. 

If we encounter a right character and the stack is empty or the top of the stack is a 
different type of left character, the right character is not matched, implying the string 
is not matched. If all characters have been processed and the stack is nonempty, there 
are unmatched left characters so the string is not matched. 

public static boolean isWellFormed(String s) { 

Deque<Character> leftChars = new LinkedList<>(); 
for (int i = Q; i < s.lengthO; ++i) { 

if (s.charAt(i) == ’(’ || s.charAt(i) == ’{’ II s.charAt(i) == ’[’) { 

leftChars.addFirst(s.charAt(i)); 

} else { 

if (leftChars . isEmptyO) { 

return false; // Unmatched right char. 

} 

if ((s.charAt(i) == ’)’ && leftChars.peekFirst() != ’(’) 

|| (s.charAt(i) == ’}’ && leftChars.peekFirst() != ’{’) 

|| (s.charAt(i) == ’]’ && leftChars.peekFirst() != ’[’)) { 

return false; // Mismatched chars. 

} 

leftChars.removeFirst(); 

} 

} 

return leftChars.isEmpty(); 

} 


The time complexity is 0(n) since for each character we perform 0(1) operations. 
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9.4 Normalize pathnames 


A file or directory can be specified via a string called the pathname. This string may 
specify an absolute path, starting from the root, e.g., /usr/bin/gcc, or a path relative 
to the current working directory, e.g., scripts/awkscripts. 

The same directory may be specified by multiple directory paths. For exam¬ 
ple, /usr/lib/. ./bin/gcc and scripts//./, ./scripts/awkscripts/./. / specify 
equivalent absolute and relative pathnames. 

Write a program which takes a pathname, and returns the shortest equivalent path¬ 
name. Assume individual directories and files have names that use only alphanu¬ 
meric characters. Subdirectory names may be combined using forward slashes (/), 
the current directory (.), and parent directory (. .). 

Hint: Trace the cases. How should . and .. be handled? Watch for invalid paths. 

Solution: It is natural to process the string from left-to-right, splitting on forward 
slashes (/s). We record directory and file names. Each time we encounter a .., we 
delete the most recent name, which corresponds to going up directory hierarchy. 
Since names are processed in a last-in, first-out order, it is natural to store them in a 
stack. Individual periods (. s) are skipped. 

If the string begins with /, then we cannot go up from it. We record this in the 
stack. If the stack does not begin with /, we may encounter an empty stack when 
processing . ., which indicates a path that begins with an ancestor of the current 
working path. We need to record this in order to give the shortest equivalent path. 
The final state of the stack directly corresponds to the shortest equivalent directory 
path. 

For example, if the string is sc//./, ./tc/awk/././, the stack progression is as 
follows: (sc), (), (tc), (tc, awk). Note that we skip three .s and the / after sc/. 


public static String shortestEquivalentPath(String path) { 
if (path.equals("")) { 

throw new IllegalArgumentException("Empty string is not a legal path."); 

} 

Deque<String> pathNames = new LinkedList<>(); 

// Special case: starts with , which is an absolute path. 
if (path.startsWith("/")) { 
pathNames.addFirst("/"); 

} 

for (String token : path.split("/")) { 
if (token.equals("..")) { 

if (pathNames.isEmpty() || pathNames.peekFirst().equals("..")) { 

pathNames.addFirst(token); 

} else { 

if (pathNames.peekFirst().equals("/")) { 
throw new IllegalArgumentException( 

"Path error, trying to go up root " + path); 

} 

pathNames.removeFirst(); 

} 
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} else if (!token.equals&& !token.isEmpty()) { // Must be a name. 
pathNames.addFirst(token); 

} 


StringBuilder result = new StringBuilder(); 
if (!pathNames.isEmpty()) { 

Iterator<String> it = pathNames.descendinglterator(); 
String prev = it.next(); 
result.append(prev); 
while (it.hasNext()) { 
if (!prev.equalsC"/")) { 
result.append("/") ; 

} 

prev = it.next() ; 
result.append(prev) ; 

} 

} 

return result.toString() ; 


The time complexity is 0(ri) r where n is the length of the pathname. 

9.5 Search a postings list 

A postings list is a singly linked list with an additional "jump" field at each node. 
The jump field points to any other node. Figure 9.3 illustrates a postings list with 
four nodes. 



One way to enumerate the nodes in a postings list is to iteratively follow the next 
field. Another is to always first follow the jump field if it leads to a node that has 
not been explored previously, and then search from the next node. Call the order in 
which these nodes are traversed the jump-first order. 

Write recursive and iterative routines that take a postings list, and compute the jump- 
first order. Assume each node has an integer-valued field that holds the order, and is 
initialized to -1. 

Hint: Recursion makes the problem trivial. Mimic the recursion with a stack. 

Solution: The recursive algorithm directly follows the specification. If the current 
node is unvisited, update the current node's order, visit its jump node, then visit the 
next node. 
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public static void setJumpOrder(PostingListNode L) { 
setJumpOrderHelper(L, ®) ; 

} 

private static int setJumpOrderHelper(PostingListNode L, int order) { 
if (L != null && L.order == -1) { 

L.order = order++; 

order = setJumpOrderHelper(L.jump, order); 
order = setJumpOrderHelper(L.next, order); 

} 

return order; 


The iterative solution uses a stack to simulate the recursive algorithm. The key 
insight is that for every node, we want to visit its next node after visiting its jump 
node. A stack works well because of its last-in, first-out semantics. Specifically, when 
processing a node, we push its next node on to the stack and then we push its jump 
node on to the stack. This way we process the jump node before the next node. 

public static void setJumpOrder(PostingListNode L) { 

Deque<PostingListNode> s = new LinkedList<>(); 
int order = Q; 
s.addFirst(L); 
while (!s.isEmpty()) { 

PostingListNode curr = s.removeFirst(); 
if (curr != null && curr.order == -1) { 
curr.order = order++; 

// Stack is last-in, first-out, and we want to process 
// the jump node first, so push next, then push jump, 
s .addFirst(curr.next); 
s.addFirst(curr.jump); 

} 

} 

} 


Let n denote the number of tree nodes. For both algorithms, the time complexity 
is 0(n), since the total time spent on each node is 0(1), and the space complexity is 
0(n). The recursive implementation has a maximum function call stack depth of n; 
the iterative implementation has a maximum stack size of n. A worst-case input is 
one where every node's jump node and next node are equal. 


9.6 Compute buildings with a sunset view 

You are given with a series of buildings that have windows facing west. The buildings 
are in a straight line, and any building which is to the east of a building of equal or 
greater height cannot view the sunset. 

Design an algorithm that processes buildings in east-to-west order and returns the 
set of buildings which view the sunset. Each building is specified by its height. 

Hint: When does a building not have a sunset view? 
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Solution: A brute-force approach is to store all buildings in an array. We then do 
a reverse scan of this array, tracking the running maximum. Any building whose 
height is less than or equal to the running maximum does not have a sunset view. 
The time and space complexity are both 0{n), where n is the number of buildings. 
Note that if a building is to the east of a taller building, it cannot view the sunset. 
This suggests a way to reduce the space complexity. We record buildings which 
potentially have a view. Each new building may block views from the existing set. 
We determine which such buildings are blocked by comparing the new building's 
height to that of the buildings in the existing set. We can store the existing set as a 
hash set—this requires us to iterate over all buildings each time a new building is 
processed. 

If a new building is shorter than a building in the current set, then all buildings in 
the current set which are further to the east cannot be blocked by the new building. 
This suggests keeping the buildings in a last-in, first-out manner, so that we can 
terminate earlier. 

Specifically, we use a stack to record buildings that have a view. Each time a 
building b is processed, if it is taller than the building at the top of the stack, we pop 
the stack until the top of the stack is taller than b —all the buildings thus removed lie 
to the east of a taller building. 

Although some individual steps may require many pops, each building is pushed 
and popped at most once. Therefore, the run time to process n buildings is 0(n), and 
the stack always holds precisely the buildings which currently have a view. 

The memory used is 0(n), and the bound is tight, even when only one building 
has a view—consider the input where the west-most building is the tallest, and the 
remaining n — 1 buildings decrease in height from east to west. However, in the 
best-case, e.g., when buildings appear in increasing height, we use (9(1) space. In 
contrast, the brute-force approach always uses 0(n) space. 

private static class BuildingWithHeight { 
public Integer id; 
public Integer height; 

public BuildingWithHeight(Integer id, Integer height) { 
this.id = id; 
this.height = height; 

} 

} 

public static Deque<BuildingWithHeight> examineBuildingsWithSunset( 

Iterator<Integer> sequence) { 
int buildingldx = Q; 

Deque<BuildingWithHeight> buildingsWithSunset = new LinkedList<>(); 
while (sequence.hasNext()) { 

Integer buildingHeight = sequence.next(); 
while (!buildingsWithSunset.isEmpty() 

&& (Integer.compare(buildingHeight, 

buildingsWithSunset.getLast().height) 

>= 8 )) ( 

buildingsWithSunset.removeLast(); 

} 
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buildingsWithSunset.addLast( 

new BuildingWithHeight(buildingldx++, buildingHeight)); 

} 

return buildingsWithSunset; 


Variant: Solve the problem subject to the same constraints when buildings are pre¬ 
sented in west-to-east order. 

Queues 

A queue supports two basic operations—enqueue and dequeue. (If the queue is 
empty, dequeue typically returns null or throws an exception.) Elements are added 
(enqueued) and removed (dequeued) in first-in, first-out order. The most recently 
inserted element is referred to as the tail or back element, and the item that was 
inserted least recently is referred to as the head or front element. 

A queue can be implemented using a linked list, in which case these operations 
have <9(1) time complexity. The queue API often includes other operations, e.g., a 
method that returns the item at the head of the queue without removing it, a method 
that returns the item at the tail of the queue without removing it, etc. A queue can 
also be implemented using an array; see Problem 9.8 on Page 145 for details. 

- dequeued - - 

3 1 2 1 0 1 v —2 1 0 1 2 1 0 1 4 1 

_I_I_I_ _I_I_ _I_I_L 

(a) Initial configuration. (b) Queue (a) after dequeue. (c) Queue (b) after enqueuing 4. 

Figure 9.4: Examples of enqueuing and dequeuing. 

A deque, also sometimes called a double-ended queue, is a doubly linked list in 
which all insertions and deletions are from one of the two ends of the list, i.e., at the 
head or the tail. An insertion to the front is commonly called a push, and an insertion 
to the back is commonly called an inject. A deletion from the front is commonly called 
a pop, and a deletion from the back is commonly called an eject. (Different languages 
and libraries may have different nomenclature.) 

Queues boot camp 

In the following program, we implement the basic queue API—enqueue and 
dequeue—as well as a max-method, which returns the maximum element stored 
in the queue. The basic idea is to use composition: add a private field that references 
a library queue object, and forward existing methods (enqueue and dequeue in this 
case) to that object. 

public class QueueWithMaxIntro { 

private Deque<Integer> data = new LinkedList<>(); 

public void enqueue(Integer x) { data.add(x); } 

public Integer dequeue() { return data.removeFirst(); } 


-u 


enqueue 4 
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public Integer max() { 
if (!data.isEmpty()) { 

return Collections.max(data); 

} 

throw new IllegalStateException("Cannot perform max() on empty queue."); 

} 

} 


The time complexity of enqueue and dequeue are the same as that of the library 
queue, namely, 0(1). The time complexity of finding the maximum is 0(n), where n 
is the number of entries. In Solution 9.10 on Page 147 we show how to improve the 
time complexity of maximum to 0(1) with a more customized approach. 


Learn to recognize when the queue FIFO property is applicable. For example, 
queues are ideal when order needs to be preserved. [Problem 9.7] 


Know your queue libraries 

The preferred way to manipulate queues is via the Deque interface. The LinkedList 
class is a doubly linked list that implements this interface, and provides efficient (<9(1) 
time) queue (and stack) functionality. 

The key queue-related methods in the Deque interface are addLast(3.14), 
removeFirstO, getFirstO, offerLast(3.14), pollFirstO, and peekFirst(). 

• addLast (3.14) enequeues an element. Some classes implementing Deque have 
capacity limits and/or preclude null from being enqueued, but this is not the 
case for LinkedList. 

• removeFirst () retrieves and removes the first element of this deque, throwing 
NoSuchElementException if the deque is empty. 

• getFirstO retrieves, but does not remove, the first element of this deque, 
throwing NoSuchElementException if the deque is empty. 

The methods offerLast(’a’), pollFirstO, and peekFirstO are very similar to 
addLastC’a’), removeFirstO, and getFirstO, the exception being that they are 
less prone to throwing exceptions. For example, if the queue is at full capac¬ 
ity, offerLast(’a’) returns false, indicating the enqueue was unsuccessful. Both 
pollFirstO and peekFirstO return null if the queue is empty—this can be am¬ 
biguous if null is a valid entry. 

Other useful queue-related methods are descendinglteratorO and iterator(). 
Deque is a subinterface of Queue. In particular, the Queue methods add(123), 
offer (123), remove (), poll(), element O, and peek() are identical to addLast (123), 
offerLastO, removeFirstO, pollFirstO, getFirstO, and peekFirstO, respec¬ 
tively. We prefer the latter methods as we find their naming more in keeping with 
the Deque concept. 

9.7 Compute binary tree nodes in order of increasing depth 

Binary trees are formally defined in Chapter 10. In particular, each node in a binary 
tree has a depth, which is its distance from the root. 
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Given a binary tree, return an array consisting of the keys at the same level. Keys 
should appear in the order of the corresponding nodes' depths, breaking ties from left 
to right. For example, you should return ((314), (6,6), (271,561,2,271), (28,0,3,1,28), 
(17,401,257), (641)) for the binary tree in Figure 10.1 on Page 150. 

Hint: First think about solving this problem with a pair of queues. 

Solution: A brute force approach might be to write the nodes into an array while 
simultaneously computing their depth. We can use preorder traversal to compute 
this array—by traversing a node's left child first we can ensure that nodes at the same 
depth are sorted from left to right. Now we can sort this array using a stable sorting 
algorithm with node depth being the sort key. The time complexity is dominated by 
the time taken to sort, i.e., 0(n log n), and the space complexity is 0(n), which is the 
space to store the node depths. 

Intuitively, since nodes are already presented in a somewhat ordered fashion in the 
tree, it should be possible to avoid a full-blow sort, thereby reducing time complexity. 
Furthermore, by processing nodes in order of depth, we do not need to label every 
node with its depth. 

In the following, we use a queue of nodes to store nodes at depth i and a queue 
of nodes at depth i + 1. After all nodes at depth i are processed, we are done with 
that queue, and can start processing the queue with nodes at depth i + 1, putting the 
depth i + 2 nodes in a new queue. 

public static List<List<Integer» binaryTreeDepthOrder( 

BinaryTreeNode<Integer> tree) { 

Queue<BinaryTreeNode<Integer>> currDepthNodes = new LinkedList<>(); 
currDepthNodes.add(tree); 

List<List<Integer>> result = new ArrayList<>(); 

while (! currDepthNodes . isEmptyO) { 

Queue<BinaryTreeNode<Integer» nextDepthNodes = new LinkedList<>(); 

Listdnteger> thisLevel = new ArrayList<>() ; 
while (! currDepthNodes . isEmptyO) { 

BinaryTreeNode<Integer> curr = currDepthNodes.poll(); 
if (curr != null) { 

thisLevel.add(curr.data); 

// Defer the null checks to the null test above. 
nextDepthNodes.add(curr.left); 
nextDepthNodes.add(curr.right); 

} 

} 

if (!thisLevel.isEmpty()) { 
result.add(thisLevel); 

} 

currDepthNodes = nextDepthNodes; 

} 

return result; 

} 


Since each node is enqueued and dequeued exactly once, the time complexity is 0(n). 
The space complexity is 0{m), where m is the maximum number of nodes at any 
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single depth. 


Variant: Write a program which takes as input a binary tree and returns the keys in 
top down, alternating left-to-right and right-to-left order, starting from left-to-right. 
For example, if the input is the tree in Figure 10.1 on Page 150, your program should 
return «314>, <6,6), <271,561,2,271), <28,1,3,0,28), <17,401,257), <641)). 

Variant: Write a program which takes as input a binary tree and returns 
the keys in a bottom up, left-to-right order. For example, if the in¬ 
put is the tree in Figure 10.1 on Page 150, your program should return 
«641), <17,401,257), <28,0,3,1,28), <271,561,2,271), <6,6), <314)). 

Variant: Write a program which takes as input a binary tree with integer keys, and 
returns the average of the keys at each level. For example, if the input is the tree in 
Figure 10.1 on Page 150, your program should return <314,6,276.25,12,225,641). 


9.8 Implement a circular queue 

A queue can be implemented using an array and two additional fields, the beginning 
and the end indices. This structure is sometimes referred to as a circular queue. Both 
enqueue and dequeue have <9( 1) time complexity. If the array is fixed, there is a 
maximum number of entries that can be stored. If the array is dynamically resized, 
the total time for m combined enqueue and dequeue operations is <9(ra). 

Implement a queue API using an array for storing elements. Your API should include 
a constructor function, which takes as argument the initial capacity of the queue, 
enqueue and dequeue functions, and a function which returns the number of elements 
stored. Implement dynamic resizing to support storing an arbitrarily large number 
of elements. 

Hint: Track the head and tail. How can you differentiate a full queue from an empty one? 

Solution: A brute-force approach is to use an array, with the head always at index 
0. An additional variable tracks the index of the tail element. Enqueue has 0(1) time 
complexity. However dequeue's time complexity is 0(n), where n is the number of 
elements in the queue, since every element has to be left-shifted to fill up the space 
created at index 0. 

A better approach is to keep one more variable to track the head. This way, 
dequeue can also be performed in <9(1) time. When performing an enqueue into a 
full array, we need to resize the array. We cannot only resize, because this results in 
queue elements not appearing contiguously. For example, if the array is (e, b, c, d), 
with e being the tail and b the head, if we resize to get (e, b, c, d, J), we cannot 

enqueue without overwriting or moving elements. 

public static class Queue { 

private int head = ®, tail = ®, numQueueElements = ®; 
private static final int SCALE_FACT0R = 2; 
private Integer[] entries; 

public Queue(int capacity) { entries = new Integer[capacity]; } 
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public void enqueue(Integer x) { 

if (numQueueElements == entries.length) { // Need to resize. 

// Makes the queue elements appear consecutively. 

Collections.rotate(Arrays.asList(entries), -head); 

// Resets head and tail . 
head = ®; 

tail = numQueueElements; 

entries = Arrays.copyOf(entries, numQueueElements * SCALE_FACTOR); 

} 

entries[tail] = x; 

tail = (tail + 1) % entries.length; 

++numQueueElements; 


public Integer dequeue() { 
if (numQueueElements != ®) { 

--numQueueElements; 

Integer ret = entries[head]; 
head = (head + 1) % entries.length; 
return ret; 

} 

throw new NoSuchElementException("Dequeue called on an empty queue."); 

} 

public int sizeQ { return numQueueElements; } 


The time complexity of dequeue is <9(1), and the amortized time complexity of en¬ 
queue is <9(1). 


9.9 Implement a queue using stacks 

Queue insertion and deletion follows first-in, first-out semantics; stack insertion and 
deletion is last-in, first-out. 

How would you implement a queue given a library implementing stacks? 

Hint: It is impossible to solve this problem with a single stack. 

Solution: A straightforward implementation is to enqueue by pushing the element 
to be enqueued onto one stack. The element to be dequeued is then the element 
at the bottom of this stack, which can be achieved by first popping all its elements 
and pushing them to another stack, then popping the top of the second stack (which 
was the bottom-most element of the first stack), and finally popping the remaining 
elements back to the first stack. 

The primary problem with this approach is that every dequeue takes two pushes 
and two pops of every element, i.e., dequeue has <9(n) time complexity, where n is the 
number of stored elements. (Enqueue takes <9(1) time.) 

The intuition for improving the time complexity of dequeue is that after we move 
elements from the first stack to the second stack, any further dequeues are trivial, 
until the second stack is empty. This is true even if we need to enqueue, as long as we 
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enqueue onto the first stack. When the second stack becomes empty, and we need to 
perform a dequeue, we simply repeat the process of transferring from the first stack to 
the second stack. In essence, we are using the first stack for enqueue and the second 
for dequeue. 

public static class Queue { 

private Deque<Integer> enq = new LinkedList<>(); 
private Deque<Integer> deq = new LinkedList<>(); 

public void enqueue(Integer x) { enq.addFirst(x); } 

public Integer dequeue() { 
if (deq.isEmpty()) { 

// Transfers the elements from enq to deq. 
while (!enq.isEmpty()) { 

deq.addFirst(enq.removeFirst()); 

} 

} 

if (!deq.isEmpty()) { 

return deq.removeFirst(); 

} 

throw new NoSuchElementException("Cannot pop empty queue"); 

} 

} 

This approach takes 0(m) time for m operations, which can be seen from the fact that 
each element is pushed no more than twice and popped no more than twice. 

9.10 Implement a queue with max API 

Implement a queue with enqueue, dequeue, and max operations. The max operation 
returns the maximum element currently stored in the queue. 

Hint: When can an element never be returned by max, regardless of future updates? 

Solution: A brute-force approach is to track the current maximum. The current 
maximum has to be updated on both enqueue and dequeue. Updating the current 
maximum on enqueue is trivial and fast—just compare the enqueued value with the 
current maximum. However, updating the current maximum on dequeue is slow— 
we must examine every single remaining element, which takes 0(n) time, where n is 
the size of the queue. 

Consider an element s in the queue that has the property that it entered the queue 
before a later element, b, which is greater than s. Since s will be dequeued before b, s 
can never in the future become the maximum element stored in the queue, regardless 
of the subsequent enqueues and dequeues. 

The key to a faster implementation of a queue-with-max is to eliminate elements 
like s from consideration. We do this by maintaining the set of entries in the queue 
that have no later entry in the queue greater than them in a separate deque. Elements 
in the deque will be ordered by their position in the queue, with the candidate closest 
to the head of the queue appearing first. Since each entry in the deque is greater than 
or equal to its successors, the largest element in the queue is at the head of the deque. 
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We now briefly describe how to update the deque on queue updates. If the queue 
is dequeued, and if the element just dequeued is at the deque's head, we pop the 
deque from its head; otherwise the deque remains unchanged. When we add an 
entry to the queue, we iteratively evict from the deque's tail until the element at the 
tail is greater than or equal to the entry being enqueued, and then add the new entry 
to the deque's tail. These operations are illustrated in Figure 9.5. 
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Figure 9.5: The queue with max for the following operations: enqueue 1 , dequeue, dequeue, enqueue 2, 
enqueue 4, dequeue, enqueue 4. The queue initially contains 3,1,3,2, and 0 in that order. The deque 
D corresponding to queue O is immediately below Q. The progression is shown from left-to-right, then 
top-to-bottom. The head of each queue and deque is on the left. Observe how the head of the deque 
holds the maximum element in the queue. 


public static class QueueWithMaxcT extends Comparable<T» { 
private Queue<T> entries = new LinkedList<>(); 
private Deque<T> candidatesForMax = new LinkedList<>(); 

public void enqueue(T x) { 
entries.add(x); 

while ( ! candidatesForMax . isEmptyO) { 

// Eliminate dominated elements in candidatesForMax. 
if (candidatesForMax.getLast().compareTo(x) >= ©) { 
break; 

} 

candidatesForMax.removeLast(); 

} 

candidatesForMax.addLast(x) ; 

} 

public T dequeue() { 

if (!entries.isEmpty()) { 

T result = entries.remove(); 

if (result.equals(candidatesForMax.getFirst())) { 
candidatesForMax.removeFirst(); 

} 

return result; 

} 

throw new NoSuchElementException("Called dequeue() on empty queue."); 


public T max() { 

if (!candidatesForMax.isEmpty()) { 
return candidatesForMax.getFirst(); 

} 

throw new NoSuchElementException("empty queue"); 
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} 


Each dequeue operation has time 0(1) complexity. A single enqueue operation may 
entail many ejections from the deque. However, the amortized time complexity of n 
enqueues and dequeues is 0(n), since an element can be added and removed from 
the deque no more than once. The max operation is 0(1) since it consists of returning 
the element at the head of the deque. 

An alternate solution that is often presented is to use reduction. Specifically, we 
know how to solve the stack-with-max problem efficiently (Solution 9.1 on Page 132) 
and we also know how to efficiently model a queue with two stacks (Solution 9.9 on 
Page 146), so we can solve the queue-with-max design by modeling a queue with 
two stacks-with-max. This approach feels unnatural compared to the one presented 
above. 

public class QueueWithMax { 

private StackWithMax.Stack enqueue = new StackWithMax.Stack(); 
private StackWithMax.Stack dequeue = new StackWithMax.Stack(); 

public void enqueue(Integer x) { enqueue.push(x); } 

public Integer dequeue() { 
if (dequeue.empty()) { 

while (!enqueue.empty()) { 
dequeue.push(enqueue.pop()); 

} 

} 

if (!dequeue.empty()) { 
return dequeue.pop() ; 

> 

throw new NoSuchElementException("Cannot get dequeue() on empty queue."); 

> 

public Integer max() { 
if (!enqueue.empty()) { 

return dequeue.empty() ? enqueue.max() 

: Math.max(enqueue.max(), dequeue.max()); 

} 

if (!dequeue.empty()) { 
return dequeue.max(); 

} 

throw new NoSuchElementException("Cannot get max() on empty queue."); 

} 

} 


Since the stack-with-max has 0(1) amortized time complexity for push, pop, and max, 
and the queue from two stacks has 0(1) amortized time complexity for enqueue and 
dequeue, this approach has <9(1) amortized time complexity for enqueue, dequeue, 
and max. 
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Chapter 


Binary Trees 

The method of solution involves the development of a theory of finite 
automata operating on infinite trees. 

— "Decidability of Second Order Theories and Automata on Trees," 

M. O. Rabin, 1969 


A binary tree is a data structure that is useful for representing hierarchy. Formally, 
a binary tree is either empty or a root node r together with a left binary tree and a 
right binary tree. The subtrees themselves are binary trees. The left binary tree is 
sometimes referred to as the left subtree of the root, and the right binary tree is referred 
to as the right subtree of the root. 

Figure 10.1 gives a graphical representation of a binary tree. Node A is the root. 
Nodes B and I are the left and right children of A. 



depth 0 
depth 1 
depth 2 
depth 3 
depth 4 
depth 5 


Figure 10.1: Example of a binary tree. The node depths range from 0 to 5. Node M has the highest 
depth (5) of any node in the tree, implying the height of the tree is 5. 


Often the node stores additional data. Its prototype is listed as follows: 

public static class BinaryTreeNode<T> { 
public T data; 

public BinaryTreeNode<T> left, right; 


Each node, except the root, is itself the root of a left subtree or a right subtree. If 
/ is the root of p's left subtree, we will say / is the left child of p, and p is the parent of 
/; the notion of right child is similar. If a node is a left or a right child of p, we say 
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it is a child of p. Note that with the exception of the root, every node has a unique 
parent. Usually, but not universally, the node object definition includes a parent field 
(which is null for the root). Observe that for any node there exists a unique sequence 
of nodes from the root to that node with each node in the sequence being a child of 
the previous node. This sequence is sometimes referred to as the search path from the 
root to the node. 

The parent-child relationship defines an ancestor-descendant relationship on 
nodes in a binary tree. Specifically, a node is an ancestor of d if it lies on the search 
path from the root to d. If a node is an ancestor of d, we say d is a descendant of that 
node. Our convention is that a node is an ancestor and descendant of itself. A node 
that has no descendants except for itself is called a leaf. 

The depth of a node n is the number of nodes on the search path from the root to n, 
not including n itself. The height of a binary tree is the maximum depth of any node 
in that tree. A level of a tree is all nodes at the same depth. See Figure 10.1 on the 
facing page for an example of the depth and height concepts. 

As concrete examples of these concepts, consider the binary tree in Figure 10.1 on 
the preceding page. Node I is the parent of / and O. Node G is a descendant of B. The 
search path to L is (A, I, /, K, L). The depth of N is 4. Node M is the node of maximum 
depth, and hence the height of the tree is 5. The height of the subtree rooted at B is 3. 
The height of the subtree rooted at H is 0. Nodes D, E r H,M,N, and P are the leaves 
of the tree. 

A full binary tree is a binary tree in which every node other than the leaves has 
two children. A perfect binary tree is a full binary tree in which all leaves are at the 
same depth, and in which every parent has two children. A complete binary tree is a 
binary tree in which every level, except possibly the last, is completely filled, and all 
nodes are as far left as possible. (This terminology is not universal, e.g., some authors 
use complete binary tree where we write perfect binary tree.) It is straightforward 
to prove using induction that the number of nonleaf nodes in a full binary tree is 
one less than the number of leaves. A perfect binary tree of height h contains exactly 
2 h+1 - 1 nodes, of which 2 h are leaves. A complete binary tree on n nodes has height 
|_lg nj. A left-skewed tree is a tree in which no node has a right child; a right-skewed 
tree is a tree in which no node has a left child. In either case, we refer to the binary 
tree as being skewed. 

A key computation on a binary tree is traversing all the nodes in the tree. (Traversing 
is also sometimes called walking.) Here are some ways in which this visit can be done. 

• Traverse the left subtree, visit the root, then traverse the right subtree 
(an inorder traversal). An inorder traversal of the binary tree in Fig¬ 
ure 10.1 on the facing page visits the nodes in the following order: 
(D, C, E, B, F, H, G, A, /, L, M, K, N, /, O, P). 

• Visit the root, traverse the left subtree, then traverse the right subtree 
(a preorder traversal). A preorder traversal of the binary tree in Fig¬ 
ure 10.1 on the preceding page visits the nodes in the following order: 
(A, B, C, D, E, F, G, H, I, /, K, L, M, N, O, F). 

• Traverse the left subtree, traverse the right subtree, and then visit the 
root (a postorder traversal). A postorder traversal of the binary tree 
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in Figure 10.1 on Page 150 visits the nodes in the following order: 

(D, E, C, H, G, F, B, M, L, N, K, J, P, O, l, A). 

Let T be a binary tree of n nodes, with height h. Implemented recursively, these 
traversals have 0(n) time complexity and 0(h) additional space complexity. (The 
space complexity is dictated by the maximum depth of the function call stack.) If 
each node has a parent field, the traversals can be done with 0(1) additional space 
complexity. 

The term tree is overloaded, which can lead to confusion; see Page 352 for an 
overview of the common variants. 

Binary trees boot camp 

A good way to get up to speed with binary trees is to implement the three basic 
traversals—inorder, preorder, and postorder. 

public static void treeTraversal(BinaryTreeNodednteger> root) { 
if (root != null) { 

// Preorder: Processes the root before the traversals of left and right 
// children. 

System.out.println("Preorder: " + root.data); 
treeTraversal(root.left); 

// Inorder: Processes the root after the traversal of left child and 
// before the traversal of right child. 

System.out.println("Inorder: " + root.data); 
treeTraversal(root.right); 

// Postorder: Processes the root after the traversals of left and right 
// children. 

System.out.println("Postorder: " + root.data); 

} 

} 


The time complexity of each approach is 0(n), where n is the number of nodes in the 
tree. Although no memory is explicitly allocated, the function call stack reaches a 
maximum depth of h, the height of the tree. Therefore, the space complexity is 0(h). 
The minimum value for h is lg n (complete binary tree) and the maximum value for h 
is n (skewed tree). 

10.1 Test if a binary tree is height-balanced 

A binary tree is said to be height-balanced if for each node in the tree, the difference 
in the height of its left and right subtrees is at most one. A perfect binary tree is 
height-balanced, as is a complete binary tree. A height-balanced binary tree does not 
have to be perfect or complete—see Figure 10.2 on the facing page for an example. 

Write a program that takes as input the root of a binary tree and checks whether the 
tree is height-balanced. 

Hint: Think of a classic binary tree algorithm. 

Solution: Here is a brute-force algorithm. Compute the height for the tree rooted 
at each node x recursively. The basic computation is to compute the height for each 
node starting from the leaves, and proceeding upwards. For each node, we check if 
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Recursive algorithms are well-suited to problems on trees. Remember to include 
space implicitly allocated on the function call stack when doing space complexity 
analysis. [Problem 10.1] 

Some tree problems have simple brute-force solutions that use 0(n) space solution, 
but subtler solutions that uses the existing tree nodes to reduce space complexity 
to 0(1). [Problem 10.14] 

Consider left- and right-skewed trees when doing complexity analysis. Note that 
0(h) complexity, where h is the tree height, translates into 0 (log n) complexity for 
balanced trees, but 0(n) complexity for skewed trees. [Problem 10.12] 

If each node has a parent field, use it to make your code simpler, and to reduce 
time and space complexity. [Problem 10.10] 

It's easy to make the mistake of treating a node that has a single child as a leaf. 
[Problem 10.6] 



Figure 10.2: A height-balanced binary tree of height 4. 


the difference in heights of the left and right children is greater than one. We can store 
the heights in a hash table, or in a new field in the nodes. This entails 0(n) storage 
and 0(n) time, where n is the number of nodes of the tree. 

We can solve this problem using less storage by observing that we do not need 
to store the heights of all nodes at the same time. Once we are done with a subtree, 
all we need is whether it is height-balanced, and if so, what its height is—we do not 
need any information about descendants of the subtree's root. 

private static class BalanceStatusWithHeight { 
public boolean balanced; 
public int height; 

public BalanceStatusWithHeight (boolean balanced, int height) { 
this .balanced = balanced; 
this. height = height; 

} 

} 

public static boolean isBalanced(BinaryTreeNodednteger> tree) { 
return checkBalanced(tree).balanced; 

} 
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private static BalanceStatusWithHeight checkBalanced( 

BinaryTreeNodednteger> tree) { 
if (tree == null) { 

return new BalanceStatusWithHeight (true , -1); // Base case. 

} 

BalanceStatusWithHeight leftResult = checkBalanced(tree.left); 
if (!leftResult.balanced) { 

return leftResult; // Left subtree is not balanced. 

} 

BalanceStatusWithHeight rightResult = checkBalanced(tree.right); 
if (!rightResult.balanced) { 

return rightResult; // Right subtree is not balanced. 

} 

boolean isBalanced = Math.abs(leftResult.height - rightResult.height) <= 1; 
int height = Math.max(leftResult.height, rightResult.height) + 1; 
return new BalanceStatusWithHeight(isBalanced, height); 


The program implements a postorder traversal with some calls possibly being elim¬ 
inated because of early termination. Specifically, if any left subtree is not height- 
balanced we do not need to visit the corresponding right subtree. The function call 
stack corresponds to a sequence of calls from the root through the unique path to the 
current node, and the stack height is therefore bounded by the height of the tree, lead¬ 
ing to an 0(h) space bound. The time complexity is the same as that for a postorder 
traversal, namely 0(n). 

Variant: Write a program that returns the size of the largest subtree that is complete. 

Variant: Define a node in a binary tree to be k -balanced if the difference in the number 
of nodes in its left and right subtrees is no more than k. Design an algorithm that 
takes as input a binary tree and positive integer k, and returns a node in the binary 
tree such that the node is not k- balanced, but all of its descendants are k- balanced. 
For example, when applied to the binary tree in Figure 10.1 on Page 150, if k = 3, your 
algorithm should return Node /. 


10.2 Test if a binary tree is symmetric 

A binary tree is symmetric if you can draw a vertical line through the root and then 
the left subtree is the mirror image of the right subtree. The concept of a symmetric 
binary tree is illustrated in Figure 10.3 on the facing page. 

Write a program that checks whether a binary tree is symmetric. 

Hint: The definition of symmetry is recursive. 

Solution: We can test if a tree is symmetric by computing its mirror image and seeing 
if the mirror image is equal to the original tree. Computing the mirror image of a tree 
is as simple as swapping the left and right subtrees, and recursively continuing. The 
time and space complexity are both 0(n), where n is the number of nodes in the tree. 


154 






(a) A symmetric binary tree. (b) An asymmetric binary tree. (c) An asymmetric binary tree. 

Figure 10.3: Symmetric and asymmetric binary trees.The tree in (a) is symmetric. The tree in (b) is 
structurally symmetric, but not symmetric, because symmetry requires that corresponding nodes have 
the same keys; here C and F as well as D and G break symmetry. The tree in (c) is asymmetric because 
there is no node corresponding to D. 


The insight to a better algorithm is that we do not need to construct the mirrored 
subtrees. All that is important is whether a pair of subtrees are mirror images. As 
soon as a pair fails the test, we can short circuit the check to false. This is shown in 
the code below. 

public static boolean isSymmetric(BinaryTreeNode<Integer> tree) { 
return tree == null || checkSymmetric(tree.left, tree.right); 

} 

private static boolean checkSymmetric(BinaryTreeNode<Integer> subtree®, 

BinaryTreeNode<Integer> subtreel) { 
if (subtree® == null && subtreel == null) { 
return true; 

} else if (subtree® != null && subtreel != null) { 
return subtree®.data == subtree 1.data 

&& checkSymmetric(subtree®.left, subtree 1.right) 

&& checkSymmetric(subtree®.right, subtree 1.left); 

} 

// One subtree is empty, and the other is not. 

return false; 

} 


The time complexity and space complexity are 0(n) and 0(h), respectively, where n 
is the number of nodes in the tree and h is the height of the tree. 


10.3 Compute the lowest common ancestor in a binary tree 

Any two nodes in a binary tree have a common ancestor, namely the root. The lowest 
common ancestor (LCA) of any two nodes in a binary tree is the node furthest from 
the root that is an ancestor of both nodes. For example, the LCA of M and N in 
Figure 10.1 on Page 150 is K. 

Computing the LCA has important applications. For example, it is an essential 
calculation when rendering web pages, specifically when computing the Cascading 
Style Sheet (CSS) that is applicable to a particular Document Object Model (DOM) 
element. 
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Design an algorithm for computing the LCA of two nodes in a binary tree in which 
nodes do not have a parent field. 

Hint: When is the root the LCA? 

Solution: A brute-force approach is to see if the nodes are in different immediate 
subtrees of the root, or if one of the nodes is the root. In this case, the root must be the 
LCA. If both nodes are in the left subtree of the root, or the right subtree of the root, 
we recurse on that subtree. The time complexity is 0(n 2 ), where n is the number of 
nodes. The worst-case is a skewed tree with the two nodes at the bottom of the tree. 

The insight to a better time complexity is that we do not need to perform multiple 
passes. If the two nodes are in a subtree, we can compute the LCA directly, instead 
of simply returning a Boolean indicating that both nodes are in that subtree. The 
program below returns an object with two fields—the first is an integer indicating 
how many of the two nodes were present in that subtree, and the second is their LCA, 
if both nodes were present. 

private static class Status { 
public int numTargetNodes; 
public BinaryTreeNodednteger> ancestor; 

public Status(int numTargetNodes, BinaryTreeNode<Integer> node) { 
this.numTargetNodes = numTargetNodes; 
this.ancestor = node; 

} 

} 

public static BinaryTreeNode<Integer> LCA(BinaryTreeNode<Integer> tree, 

BinaryTreeNode<Integer> node®, 
BinaryTreeNode<Integer> nodel) { 
return LCAHelper(tree, node®, nodel).ancestor; 

} 

// Returns an object consisting of an int and a node . The int field is 
// ®, 1, or 2 depending on how many of {node®, nodel} are present in 
// the tree. If both are present in the tree, when ancestor is 
// assigned to a non-null value, it is the LCA. 
private static Status LCAHelper(BinaryTreeNode<Integer> tree, 

BinaryTreeNode<Integer> node®, 

BinaryTreeNode<Integer> nodel) { 

if (tree == null) { 

return new Status(®, null); 

} 

Status leftResult = LCAHelper(tree.left, node®, nodel); 
if (leftResult.numTargetNodes == 2) { 

// Found both nodes in the left subtree. 
return leftResult; 

} 

Status rightResult = LCAHelper(tree.right, node®, nodel); 
if (rightResult.numTargetNodes == 2) { 

// Found both nodes in the right subtree. 
return rightResult; 

} 

int numTargetNodes = leftResult.numTargetNodes + rightResult.numTargetNodes 
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+ (tree == node® ? 1 : ®) + (tree == nodel ? 1 : ®) ; 
return new Status(numTargetNodes, numTargetNodes == 2 ? tree : null); 

} 


The algorithm is structurally similar to a recursive postorder traversal, and the com¬ 
plexities are the same. Specifically, the time complexity and space complexity are 
0(n) and 0(h), respectively, where h is the height of the tree. 


10.4 Compute the LCA when nodes have parent pointers 

Given two nodes in a binary tree, design an algorithm that computes their LCA. 
Assume that each node has a parent pointer. 

Hint: The problem is easy if both nodes are the same distance from the root. 

Solution: A brute-force approach is to store the nodes on the search path from the 
root to one of the nodes in a hash table. This is easily done since we can use the parent 
field. Then we go up from the second node, stopping as soon as we hit a node in the 
hash table. The time and space complexity are both 0(h), where h is the height of the 
tree. 

We know the two nodes have a common ancestor, namely the root. If the nodes 
are at the same depth, we can move up the tree in tandem from both nodes, stopping 
at the first common node, which is the LCA. However, if they are not the same depth, 
we need to keep the set of traversed nodes to know when we find the first common 
node. We can circumvent having to store these nodes by ascending from the deeper 
node to get the same depth as the shallower node, and then performing the tandem 
upward movement. 

For example, for the tree in Figure 10.1 on Page 150, nodes M and P are depths 
5 and 3, respectively. Their search paths are (A,I,J,K,L,M) and ( A,I,0,P ). If we 
ascend to depth 3 from M, we get to K. Now when we move upwards in tandem, the 
first common node is I, which is the LCA of M and P. 

Computing the depth is straightforward since we have the parent field—the time 
complexity is 0(h) and the space complexity is 0(1). Once we have the depths we can 
perform the tandem move to get the LCA. 

public static BinaryTree<Integer> LCA(BinaryTree<Integer> node®, 

BinaryTree<Integer> nodel) { 
int depth® = getDepth(node®), depthl = getDepth(node 1); 

// Makes node <9 as the deeper node in order to simplify the code. 
if (depthl > depth®) { 

BinaryTree<Integer> temp = node®; 
node® = nodel; 
nodel = temp; 

} 

// Ascends from the deeper node. 
int depthDiff = Math.abs(depth® - depthl); 
while (depthDiff-- > ®) { 
node® = node®.parent; 

} 

// Now ascends both nodes until we reach the LCA. 


157 



while (node® != nodel) { 
node® = node®.parent; 
nodel = node 1.parent; 

} 

return node®; 


private static int getDepth(BinaryTree<Integer> node) { 
int depth = ®; 

while (node.parent != null) { 

++depth; 

node = node.parent; 

} 

return depth; 


The time and space complexity are that of computing the depth, namely 0(h) and 
(9(1), respectively. 


10.5 Sum the root-to-leaf paths in a binary tree 

Consider a binary tree in which each node contains a binary digit. A root- 
to-leaf path can be associated with a binary number—the MSB is at the 
root. As an example, the binary tree in Figure 10.4 represents the numbers 

(1000) 2 , (1001) 2 , ( 10110 ) 2 , ( 110011 ) 2 , ( 11000 ) 2 , and (1100) 2 . 



Figure 10.4: Binary tree encoding integers. 


Design an algorithm to compute the sum of the binary numbers represented by the 
root-to-leaf paths. 

Hint: Think of an appropriate way of traversing the tree. 

Solution: Here is a brute-force algorithm. We compute the leaves, and store the child- 
parent mapping in a hash table, e.g., via an inorder walk. Afterwards, we traverse 
from each of the leaves to the root using the child-parent map. Each leaf-to-root path 
yields a binary integer, with the leaf's bit being the LSB. We sum these integers to 
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obtain the result. The time complexity is 0(Lh), where L is the number of root-to- 
leaf paths (which equals the number of leaves), and h is the tree height. The space 
complexity is dominated by the hash table, namely 0(n), where n is the number of 
nodes. 

The insight to improving complexity is to recognize that paths share nodes and 
that it is not necessary to repeat computations across the shared nodes. To compute 
the integer for the path from the root to any node, we take the integer for the node's 
parent, double it, and add the bit at that node. For example, the integer for the path 
from A to L is 2 x (1100) 2 + 1 = (11001) 2 . 

Therefore, we can compute the sum of all root to leaf node as follows. Each time 
we visit a node, we compute the integer it encodes using the number for its parent. 
If the node is a leaf we return its integer. If it is not a leaf, we return the sum of the 
results from its left and right children. 

public static int sumRootToLeaf(BinaryTreeNode<Integer> tree) { 
return sumRootToLeafHelper(tree, ®); 

} 

private static int sumRootToLeafHelper(BinaryTreeNode<Integer> tree, 

int partialPathSum) { 

if (tree == null) { 
return <9; 

} 

partialPathSum = partialPathSum * 2 + tree.data; 
if (tree.left == null && tree.right == null) { // Leaf. 
return partialPathSum; 

} 

// Non-leaf. 

return sumRootToLeafHelper(tree.left, partialPathSum) 

+ sumRootToLeafHelper(tree.right, partialPathSum); 


The time complexity and space complexity are 0(n) and 0{h), respectively. 

10.6 Find a root to leaf path with specified sum 

You are given a binary tree where each node is labeled with an integer. The path 
weight of a node in such a tree is the sum of the integers on the unique path from the 
root to that node. For the example shown in Figure 10.1 on Page 150, the path weight 
of E is 591. 

Write a program which takes as input an integer and a binary tree with integer node 
weights, and checks if there exists a leaf whose path weight equals the given integer. 

Hint: What do you need to know about the rest of the tree when checking a specific subtree? 

Solution: The brute-force algorithm in Solution 10.5 on the preceding page can be 
directly applied to this problem, and it has the same time complexity, namely, 0(Lh), 
where L is the number of root-to-leaf paths (which equals the number of leaves), and 
h is the tree height. The space complexity is dominated by the hash table, namely 
0(n), where n is the number of nodes. 
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The inefficiency in the brute-force algorithm stems from the fact that we have 
overlapping paths, and we do not share the summation computation across those 
overlaps. 

A better approach is to traverse the tree, keeping track of the root-to-node path 
sum. The first time we encounter a leaf whose weight equals the target weight, we 
have succeeded at locating a desired leaf. Short circuit evaluation of the check ensures 
that we do not process additional leaves. 

public static boolean hasPathSum(BinaryTreeNode<Integer> tree, 

int targetSum) { 

return hasPathSumHelper(tree, ®, targetSum); 

} 

private static boolean hasPathSumHelper(BinaryTreeNode<Integer> node, 

int partialPathSum, int targetSum) { 

if (node == null) { 
return false; 

} 

partialPathSum += node.data; 

if (node.left == null && node.right == null) { // Leaf. 
return partialPathSum == targetSum; 

} 

// Non-leaf. 

return hasPathSumHelper(node.left, partialPathSum, targetSum) 

|| hasPathSumHelper(node.right, partialPathSum, targetSum); 

} 


The time complexity and space complexity are 0(n) and 0(h ), respectively. 

Variant: Write a program which takes the same inputs as in Problem 10.6 on the 
preceding page and returns all the paths to leaves whose weight equals s. For 
example, if s = 619, you should return ((A, B, C, D), (A, I, O, P)). 


10.7 Implement an inorder traversal without recursion 

This problem is concerned with traversing nodes in a binary tree in an inorder fashion. 
See Page 151 for details and examples of these traversals. Generally speaking, a 
traversal computation is easy to implement if recursion is allowed. 

Write a program which takes as input a binary tree and performs an inorder traversal 
of the tree. Do not use recursion. Nodes do not contain parent references. 

Hint: Simulate the function call stack. 

Solution: The recursive solution is trivial—first traverse the left subtree, then visit 
the root, and finally traverse the right subtree. This algorithm can be converted into a 
iterative algorithm by using an explicit stack. Several implementations are possible; 
the one below is noteworthy in that it pushes the current node, and not its right child. 
Furthermore, it does not use a visited field. 

public static List<Integer> BSTInSortedOrder(BSTNode<Integer> tree) { 
Deque<BSTNode<Integer>> s = new LinkedList<>(); 

BSTNode<Integer> curr = tree; 
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List<Integer> result = new ArrayList<>(); 


while (!s.isEmpty() || curr != null) { 

if (curr != null) { 
s.addFirst(curr) ; 

// Going left. 
curr = curr.left; 

} else { 

// Going up. 

curr = s.removeFirst(); 

result.add(curr.data); 

// Going right. 
curr = curr.right; 

} 

} 

return result; 


For the binary tree in Figure 10.1 on Page 150, the first few stack states are (A), (A, B), 
<A, B, C>, (A, B, C, D>, (A, B, C>, {A, B, D), {A, B>, (A), (A, F). 

The time complexity is 0(n), since the total time spent on each node is (9(1). The 
space complexity is 0(h), where h is the height of the tree. This space is allocated 
dynamically, specifically it is the maximum depth of the function call stack for the 
recursive implementation. See Page 151 for a definition of tree height. 


10.8 Implement a preorder traversal without recursion 

This problem is concerned with traversing nodes in a binary tree in preorder fashion. 
See Page 151 for details and examples of these traversals. Generally speaking, a 
traversal computation is easy to implement if recursion is allowed. 

Write a program which takes as input a binary tree and performs a preorder traversal 
of the tree. Do not use recursion. Nodes do not contain parent references. 

Solution: 

We can get intuition as to the best way to perform a preorder traversal without 
recursion by noting that a preorder traversal visits nodes in a last in, first out order. We 
can perform the preorder traversal using a stack of tree nodes. The stack is initialized 
to contain the root. We visit a node by popping it, adding first its right child, and then 
its left child to the stack. (We add the left child after the right child, since we want to 
continue with the left child.) 

For the binary tree in Figure 10.1 on Page 150, the first few stack states are (A), 
( I,B >, (I, F, C>, <I,F,E,D>, ( I,F,E >, (I,F), <I,G>, </,H>, and (I). (The top of the stack is 
the rightmost node in the sequences.) 

public static List<Integer> preorderTraversal(BinaryTreeNode<Integer> tree) { 
Deque<BinaryTreeNode<Integer>> path = new LinkedList<>(); 
path.addFirst(tree); 

Listdnteger> result = new ArrayList<>() ; 
while (!path.isEmpty()) { 

BinaryTreeNode<Integer> curr = path.removeFirst(); 
if (curr != null) { 
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result.add(curr.data); 
path.addFirst(curr.right) ; 
path.addFirst(curr.left); 

} 

} 

return result; 


Since we push and pop each node exactly once, the time complexity is 0(n), where 
n is the number of nodes. The space complexity is 0(h), where h is the height of the 
tree, since, with the possible exception of the top of the stack, the nodes in the stack 
correspond to the right children of the nodes on a path beginning at the root. 

10.9 Compute the /cth node in an inorder traversal 

It is trivial to find the /cth node that appears in an inorder traversal with 0(n) time 
complexity, where n is the number of nodes. However, with additional information 
on each node, you can do better. 

Write a program that efficiently computes the /cth node appearing in an inorder 
traversal. Assume that each node stores the number of nodes in the subtree rooted at 
that node. 

Hint: Use the divide and conquer principle. 

Solution: The brute-force approach is to perform an inorder walk, keeping track of 
the number of visited nodes, stopping when the node being visited is the /cth one. 
The time complexity is 0(n). (Consider for example, a left-skewed tree—to get the 
first node (k = 1) we have to pass through all the nodes.) 

Looking carefully at the brute-force algorithm, observe that it does not take ad¬ 
vantage of the information present in the node. For example, if k is greater than the 
number of nodes in the left subtree, the /cth node cannot lie in the left subtree. More 
precisely, if the left subtree has L nodes, then the /cth node in the original tree is the 
(k - L)th node when we skip the left subtree. Conversely, if k < L, the desired node 
lies in the left subtree. For example, the left subtree in Figure 10.1 on Page 150 has 
seven nodes, so the tenth node cannot in the left subtree. Instead it is the third node 
if we skip the left subtree. This observation leads to the following program. 

public static BinaryTreeNode<Integer> findKthNodeBinaryTree( 

BinaryTreeNode<Integer> tree, int k) { 

BinaryTreeNode<Integer> iter = tree; 
while (iter != null) { 

int leftSize = iter.left != null ? iter.left.size : ®; 

if (leftSize + 1 < k) { // ic-th node must be in right subtree of iter. 
k -= (leftSize + 1); 
iter = iter.right; 

} else if (leftSize == k - 1) { // k-th is iter itself. 
return iter; 

} else { // ic-th node must be in left subtree of iter. 
iter = iter.left; 

} 

} 
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// If k is between 1 and the tree size, this line is unreachable. 

return null; 


Since we descend the tree in each iteration, the time complexity is 0(h ), where h is the 
height of the tree. 


10.10 Compute the successor 

The successor of a node in a binary tree is the node that appears immediately after 
the given node in an inorder traversal. For example, in Figure 10.1 on Page 150, the 
successor of G is A, and the successor of A is /. 

Design an algorithm that computes the successor of a node in a binary tree. Assume 
that each node stores its parent. 

Hint: Study the node's right subtree. What if the node does not have a right subtree? 

Solution: The brute-force algorithm is to perform the inorder walk, stopping imme¬ 
diately at the first node to be visited after the given node. The time complexity is that 
of an inorder walk, namely 0(n), where n is the number of nodes. 

Looking more carefully at the structure of the tree, observe that if the given node 
has a nonempty right subtree, its successor must lie in that subtree, and the rest of 
the nodes are immaterial. For example, in Figure 10.1 on Page 150, regardless of the 
structure of A's left subtree, A's successor must lie in the subtree rooted at J. Similarly, 
B's successor must lie in the subtree rooted at F. Furthermore, when a node has a 
nonempty right subtree, its successor is the first node visited when performing an 
inorder traversal on that subtree. This node is the "left-most" node in that subtree, 
and can be computed by following left children exclusively, stopping when there is 
no left child to continue from. 

The challenge comes when the given node does not have a right subtree, e.g., H 
in Figure 10.1 on Page 150. If the node is its parent's left child, the parent will be the 
next node we visit, and hence is its successor, e.g., G is H's successor. If the node 
is its parent's right child, e.g., G, then we have already visited the parent. We can 
determine the next visited node by iteratively following parents, stopping when we 
move up from a left child. For example, from G we traverse F, then B, then A. We 
stop at A, since B is the left child of A —the successor of G is A. 

Note that we may reach the root without ever moving up from a left child. This 
happens when the given node is the last node visited in an inorder traversal, and 
hence has no successor. Node P in Figure 10.1 on Page 150 illustrates this scenario. 

public static BinaryTree<Integer> findSuccessor(BinaryTree<Integer> node) { 
BinaryTree<Integer> iter = node; 
if (iter.right != null) { 

// Find the leftmost element in node's right subtree. 
iter = iter.right; 
while (iter.left != null) { 
iter = iter.left; 

} 

return iter; 
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} 

// Find the closest ancestor whose left subtree contains node. 
while (iter.parent != null <&& iter.parent.right == iter) { 
iter = iter.parent; 

} 

// A return value of null means node does not have successor, i.e., it is 
// the rightmost node in the tree. 
return iter.parent; 


Since the number of edges followed cannot be more than the tree height, the time 
complexity is 0(h), where h is the height of the tree. 


10.11 Implement an inorder traversal with 0(1) space 

The direct implementation of an inorder traversal using recursion has 0(h) space 
complexity, where h is the height of the tree. Recursion can be removed with an 
explicit stack, but the space complexity remains 0(h). 

Write a nonrecursive program for computing the inorder traversal sequence for a 
binary tree. Assume nodes have parent fields. 

Hint: How can you tell whether a node is a left child or right child of its parent? 

Solution: The standard idiom for an inorder traversal is traverse-left, visit-root, 
traverse-right. When we complete traversing a subtree we need to return to its 
parent. What we do after that depends on whether the subtree we returned from was 
the left subtree or right subtree of the parent. In the former, we visit the parent, and 
then its right subtree; in the latter, we return from the parent itself. 

One way to do this traversal without recursion is to record the parent node for 
each node we begin a traversal from. This can be done with a hash table, and entails 
0(n) time and space complexity for the hash table, where n is the number of nodes, 
and h the height of the tree. The space complexity can be reduced to 0(h) by evicting 
a node from the hash table when we complete traversing the subtree rooted at it. 

For the given problem, since each node stores its parent, we do not need the hash 
table, which improves the space complexity to 0(1). 

To complete this algorithm, we need to know when we return to a parent if the 
just completed subtree was the parent's left child (in which case we need to visit 
the parent and then traverse its right subtree) or a right subtree (in which case we 
have completed traversing the parent). We achieve this by recording the subtree's 
root before we move to the parent. We can then compare the subtree's root with the 
parent's left child. For example, for the tree in Figure 10.1 on Page 150, after traversing 
the subtree rooted at C, when we return to B, we record C. Since C is B's left child, we 
still need to traverse B's right child. When we return from F to B, we record F. Since 
F is not B's left child, it must be B's right child, and we are done traversing B. 

public static List<Integer> inorderTraversal(BinaryTree<Integer> tree) { 
BinaryTree<Integer> prev = null, curr = tree; 

Listdnteger> result = new ArrayList<>() ; 
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while (curr != null) { 

BinaryTree<Integer> next; 
if (curr.parent == prev) { 

// Fife came down to curr from prev. 
if (curr.left != null) { // Keep going left. 

next = curr.left; 

} else { 

result.add(curr.data); 

// Done with left, so go right if right is not empty. 

// Otherwise, go up. 

next = (curr.right != null) ? curr.right : curr.parent; 

} 

} else if (curr.left == prev) { 
result.add(curr.data); 

// Done with left, so go right if right is not empty. Otherwise, go up. 
next = (curr.right != null) ? curr.right : curr.parent; 

} else { // Done with both children, so move up. 
next = curr.parent; 

} 

prev = curr; 
curr = next; 

} 

return result; 


The time complexity is 0(n) and the additional space complexity is 0(1). 

Alternatively, since the successor of a node is the node appearing after it in an 
inorder visit sequence, we could start with the left-most node, and keep calling 
successor. This enables us to reuse Solution 10.10 on Page 163. 

Variant: How would you perform preorder and postorder traversals iteratively using 
0(1) additional space? Your algorithm cannot modify the tree. Nodes have an explicit 
parent field. 


10.12 Reconstruct a binary tree from traversal data 

Many different binary trees yield the same sequence of keys in an inorder, preorder, 
or postorder traversal. However, given an inorder traversal and one of any two other 
traversal orders of a binary tree, there exists a unique binary tree that yields those 
orders, assuming each node holds a distinct key. For example, the unique binary 
tree whose inorder traversal sequence is (F, B, A, E, H, C, D, I, G) and whose preorder 
traversal sequence is (HfB^^EfA^QD.G^I) is given in Figure 10.5 on the following 
page. 

Given an inorder traversal sequence and a preorder traversal sequence of a binary 
tree write a program to reconstruct the tree. Assume each node has a unique key. 

Hint: Focus on the root. 
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Figure 10.5: A binary tree—edges that do not terminate in nodes denote empty subtrees. 


Solution: A truly brute-force approach is to enumerate every binary tree on the 
inorder traversal sequence, and check if the preorder sequence from that tree is the 
given one. The complexity is enormous. 

Looking more carefully at the example, the preorder sequence gives us the key of 
the root node—it is the first node in the sequence. This in turn allows us to split the 
inorder sequence into an inorder sequence for the left subtree, followed by the root, 
followed by the right subtree. 

The next insight is that we can use the left subtree inorder sequence to compute the 
preorder sequence for the left subtree from the preorder sequence for the entire tree. 
A preorder traversal sequence consists of the root, followed by the preorder traversal 
sequence of the left subtree, followed by the preorder traversal sequence of the right 
subtree. We know the number k of nodes in the left subtree from the location of the 
root in the inorder traversal sequence. Therefore, the subsequence of k nodes after 
the root in the preorder traversal sequence is the preorder traversal sequence for the 
left subtree. 

As a concrete example, for the inorder traversal sequence (F,B,A,E,H, C,D,l, G) 
and preorder traversal sequence (H, B, F, E, A, C, D, G, I } (in Figure 10.5) the root is the 
first node in the preorder traversal sequence, namely H. From the inorder traver¬ 
sal sequence, we know the inorder traversal sequence for the root's left subtree is 
(.F,B,A,E ). Therefore the sequence ( B,F,E,A >, which is the four nodes after the root, 
H, in the preorder traversal sequence (H, B, F, E, A, C, D, G, I) is the preorder traversal 
sequence for the root's left subtree. A similar construction applies to the root's right 
subtree. This construction is continued recursively till we get to the leaves. 

Implemented naively, the above algorithm has a time complexity of 0(n 2 ). The 
worst-case corresponds to a skewed tree. Finding the root within the inorder se¬ 
quences takes time 0{n). We can improve the time complexity by initially building a 
hash table from keys to their positions in the inorder sequence. This is the approach 
described below. 

public static BinaryTreeNode<Integer> binaryTreeFromPreorderlnorder( 

List<Integer> preorder, Listdnteger> inorder) { 

Mapdnteger , Integer> nodeToInorderldx = new HashMapcInteger , Integer>() ; 
for (int i = Q; i < inorder. size () ; ++i) { 
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nodeToInorderldx.put(inorder.get(i), i); 

} 

return binaryTreeFromPreorderlnorderHelper( 

preorder, ®, preorder.size(), ®, inorder.size(), nodeToInorderldx); 

} 

// Builds the subtree with preorder.subList(preorderStart, preorderEnd) and 

// inorder.subList(inorderStart, inorderEnd). 

private static BinaryTreeNode<Integer> binaryTreeFromPreorderlnorderHelper( 
List<Integer> preorder, int preorderStart, int preorderEnd, 
int inorderStart, int inorderEnd, 

Mapdnteger, Integer> nodeToInorderldx) { 

if (preorderEnd <= preorderStart || inorderEnd <= inorderStart) { 
return null; 

} 

int rootlnorderldx = nodeToInorderldx.get(preorder.get(preorderStart)); 

int leftSubtreeSize = rootlnorderldx - inorderStart; 

return new BinaryTreeNode<>( 

preorder.get(preorderStart), 

// Recursively builds the left subtree. 
binaryTreeFromPreorderlnorderHelper( 

preorder, preorderStart + 1, preorderStart + 1 + leftSubtreeSize, 
inorderStart, rootlnorderldx, nodeToInorderldx), 

// Recursively builds the right subtree. 
binaryTreeFromPreorderlnorderHelper( 

preorder, preorderStart + 1 + leftSubtreeSize, preorderEnd, 
rootlnorderldx + 1, inorderEnd, nodeToInorderldx)); 


The time complexity is 0(n )—building the hash table takes 0(n) time and the recursive 
reconstruction spends 0(1) time per node. The space complexity is 0(n + h) = 0(n )— 
the size of the hash table plus the maximum depth of the function call stack. 

Variant: Solve the same problem with an inorder traversal sequence and a postorder 
traversal sequence. 

Variant: Let A be an array of n distinct integers. Let the index of the maximum 
element of A be m. Define the max-tree on A to be the binary tree on the entries of A 
in which the root contains the maximum element of A, the left child is the max-tree 
on A[0 : m - 1] and the right child is the max-tree on A[m + 1 : n — 1]. Design an 0(n) 
algorithm for building the max-tree of A. 


10.13 Reconstruct a binary tree from a preorder traversal with markers 

Many different binary trees have the same preorder traversal sequence. 

In this problem, the preorder traversal computation is modified to mark where a 
left or right child is empty. For example, the binary tree in Figure 10.5 on the facing 
page yields the following preorder traversal sequence: 

( H , B, F, null, null, E, A, null, null, null, C, null, D, null, G, I, null, null, null) 
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Design an algorithm for reconstructing a binary tree from a preorder traversal visit 
sequence that uses null to mark empty children. 

Hint: It's difficult to solve this problem by examining the preorder traversal visit sequence from 
left-to-right. 

Solution: One brute-force approach is to enumerate all binary trees and compare the 
resulting preorder sequence with the given one. This approach will have unacceptable 
time complexity. 

The intuition for a better algorithm is the recognition that the first node in the 
sequence is the root, and the sequence for the root's left subtree appears before all the 
nodes in the root's right subtree. It is not easy to see where the left subtree sequence 
ends. However, if we solve the problem recursively, we can assume that the routine 
correctly computes the left subtree, which will also tell us where the right subtree 
begins. 

// Global variable, tracks current subtree. 
private static Integer subtreeldx; 

public static BinaryTreeNode<Integer> reconstructPreorder( 

Listdnteger> preorder) { 
subtreeldx = ®; 

return reconstructPreorderSubtree(preorder); 

1 

// Reconstructs the subtree that is rooted at subtreeldx . 
private static BinaryTreeNode<Integer> reconstructPreorderSubtree( 

Listdnteger> preorder) { 

Integer subtreeKey = preorder.get(subtreeldx); 

++subtreeldx; 
if (subtreeKey == null) { 
return null; 

} 

// Note that reconstructPreorderSubtree updates subtreeldx. So the order of 
// following two calls are critical. 

BinaryTreeNodednteger> leftSubtree = reconstructPreorderSubtree(preorder); 
BinaryTreeNode<Integer> rightSubtree = reconstructPreorderSubtree(preorder); 
return new BinaryTreeNode<>(subtreeKey, leftSubtree, rightSubtree); 


The time complexity is 0{n), where n is the number of nodes in the tree. 

Variant: Solve the same problem when the sequence corresponds to a postorder 
traversal sequence. Is this problem solvable when the sequence corresponds to an 
inorder traversal sequence? 


10.14 Form a linked list from the leaves of a binary tree 

In some applications of a binary tree, only the leaf nodes contain actual information. 
For example, the outcomes of matches in a tennis tournament can be represented by 
a binary tree where leaves are players. The internal nodes correspond to matches, 
with a single winner advancing. For such a tree, we can link the leaves to get a list of 
participants. 
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Given a binary tree, compute a linked list from the leaves of the binary tree. The 
leaves should appear in left-to-right order. For example, when applied to the binary 
tree in Figure 10.1 on Page 150, your function should return (D, E, H, M, N, P). 

Hint: Build the list incrementally—it's easy if the partial list is a global. 

Solution: A fairly direct approach is to use two passes—one to compute the leaves, 
and the other to assign ranks to the leaves, with the left-most leaf getting the lowest 
rank. The final result is the leaves sorted by ascending order of rank. 

In fact, it is not necessary to make two passes—if we process the tree from left to 
right, the leaves occur in the desired order, so we can incrementally add them to the 
result. This idea is shown below. 

public static List<BinaryTreeNode<Integer>> createListOfLeaves( 

BinaryTreeNode<Integer> tree) { 

List<BinaryTreeNode<Integer>> leaves = new LinkedList<>(); 
addLeavesLeftToRight(tree, leaves); 
return leaves; 

} 

private static void addLeavesLeftToRight( 

BinaryTreeNode<Integer> tree, List<BinaryTreeNodednteger>> leaves) { 
if (tree != null) { 

if (tree.left == null && tree.right == null) { 
leaves.add(tree); 

} else { 

addLeavesLeftToRight(tree.left, leaves); 
addLeavesLeftToRight(tree.right, leaves); 

> 

} 


The time complexity is 0{n), where n is the number of nodes. 


10.15 Compute the exterior of a binary tree 

The exterior of a binary tree is the following sequence of nodes: the nodes from the 
root to the leftmost leaf, followed by the leaves in left-to-right order, followed by the 
nodes from the rightmost leaf to the root. (By leftmost (rightmost) leaf, we mean the 
leaf that appears first (last) in an inorder traversal.) For example, the exterior of the 
binary tree in Figure 10.1 on Page 150 is {A, B, C, D, E, H, M, N, P, O, I). 

Write a program that computes the exterior of a binary tree. 

Hint: Handle the root's left child and right child in mirror fashion. 

Solution: A brute-force approach is to use a case analysis. We need the nodes on 
the path from the root to the leftmost leaf, the nodes on the path from the root to the 
rightmost leaf, and the leaves in left-to-right order. 

We already know how to compute the leaves in left-to-right order (Solution 10.14). 
The path from root to leftmost leaf is computed by going left if a left child exists, and 
otherwise going right. When we reach a leaf, it must be the leftmost leaf. A similar 
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computation yields the nodes on the path from the root to the rightmost leaf. The 
time complexity is 0(h + n + h) = 0(n), where n and h are the number of nodes and 
the height of the tree, respectively The implementation is a little tricky, because some 
nodes appear in multiple sequences. For example, in Figure 10.1 on Page 150, the 
path from the root to the leftmost leaf is (A, B, C, D), the leaves in left-to-right order are 
(D, E, H, M, N, P), and the path from the root to the rightmost leaf is (A, I , O, P). Note 
the leftmost leaf, D, the rightmost leaf, P, and the root. A, appear in two sequences. 

We can simplify the above approach by computing the nodes on the path from the 
root to the leftmost leaf and the leaves in the left subtree in one traversal. After that, 
we find the leaves in the right subtree followed by the nodes from the rightmost leaf 
to the root with another traversal. This is the program shown below. For the tree 
in Figure 10.1 on Page 150, the first traversal returns (B,C,D,E,H), and the second 
traversal returns (M, N, P, O, I). We append the first and then the second sequences to 
<A>. 

public static List<BinaryTreeNode<Integer>> exteriorBinaryTree( 

BinaryTreeNode<Integer> tree) { 

List<BinaryTreeNodednteger>> exterior = new LinkedList<>(); 
if (tree != null) { 
exterior.add(tree); 

exterior.addAll(leftBoundaryAndLeaves(tree.left, true)) ; 
exterior.addAll(rightBoundaryAndLeaves(tree.right, true)); 

} 

return exterior; 


// Computes the nodes from the root to the leftmost leaf followed by all the 
// leaves in subtreeRoot. 

private static List<BinaryTreeNode<Integer>> leftBoundaryAndLeaves( 
BinaryTreeNode<Integer> subtreeRoot, boolean isBoundary) { 
List<BinaryTreeNode<Integer>> result = new LinkedList<>(); 
if (subtreeRoot != null) { 

if (isBoundary || isLeaf(subtreeRoot)) { 
result.add(subtreeRoot); 

} 

result.addAll(leftBoundaryAndLeaves(subtreeRoot.left, isBoundary)); 
result.addAll(leftBoundaryAndLeaves( 

subtreeRoot.right, isBoundary <&& subtreeRoot.left == null)); 

} 

return result; 


// Computes the leaves in left-to-right order followed by the rightmost leaf 
// to the root path in subtreeRoot. 

private static List<BinaryTreeNode<Integer>> rightBoundaryAndLeaves( 
BinaryTreeNode<Integer> subtreeRoot, boolean isBoundary) { 

List<BinaryTreeNode<Integer>> result = new LinkedList<>(); 
if (subtreeRoot != null) { 

result.addAll(rightBoundaryAndLeaves( 

subtreeRoot.left, isBoundary && subtreeRoot.right == null)); 
result.addAll(rightBoundaryAndLeaves(subtreeRoot.right, isBoundary)); 
if (isBoundary || isLeaf(subtreeRoot)) { 
result.add(subtreeRoot) ; 

} 
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Figure 10.6: Assigning each node’s level-next field to its right sibling in a perfect binary tree. A dashed 
arrow indicates the value held by the level-next field after the update. No dashed arrow is shown for the 
nodes on the path from the root to the rightmost leaf, i.e., A,I,M, and O, since these nodes have no 
right siblings. 


} 

return result; 

} 

private static boolean isLeaf(BinaryTreeNode<Integer> node) { 
return node.left == null && node.right == null; 

} 


The time complexity is 0(n). 


10.16 Compute the right sibling tree 

For this problem, assume that each binary tree node has a extra field, call it level-next, 
that holds a binary tree node (this field is distinct from the fields for the left and right 
children). The level-next field will be used to compute a map from nodes to their 
right siblings. The input is assumed to be perfect binary tree. See Figure 10.6 for an 
example. 

Write a program that takes a perfect binary tree, and sets each node's level-next field 
to the node on its right, if one exists. 

Hint: Think of an appropriate traversal order. 

Solution: A brute-force approach is to compute the depth of each node, which is 
stored in a hash table. Next we order nodes at the same depth using inorder visit 
times. Then we set each node's level-next field according to this order. The time and 
space complexity are 0(n), where n is the number of nodes. 

The key insight into solving this problem with better space complexity is to use 
the structure of the tree. Since it is a perfect binary tree, for a node which is a left 
child, its right sibling is just its parent's right child. For a node which is a right child, 
its right sibling is its parent's right sibling's left child. For example in Figure 10.6, 
since C is B's left child, C's right sibling is B's right child, i.e., F. Since Node F is B's 
right child, F's right sibling is B's right sibling's left child, i.e., /. 
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Figure 10.7: Assigning each node’s level-next field to its right sibling in a general binary tree. A dashed 
arrow indicates the value held by the level-next field after the update. 


For this approach to work, we need to ensure that we process nodes level-by-level, 
left-to-right. Traversing a level in which the level-next field is set is trivial. As we do 
the traversal, we set the level-next fields for the nodes on the level below using the 
principle outlined above. To get to the next level, we record the starting node for each 
level. When we complete that level, the next level is the starting node's left child. 

public static void constructRightSibling(BinaryTreeNode<Integer> tree) { 
BinaryTreeNode<Integer> leftStart = tree; 
while (leftStart != null && leftStart.left != null) { 
populateLowerLevelNextField(leftStart); 
leftStart = leftStart.left; 

} 


private static void populateLowerLevelNextField( 

BinaryTreeNode<Integer> startNode) { 

BinaryTreeNode<Integer> iter = startNode; 
while (iter != null) { 

// Populate left child’s next field. 
iter.left.next = iter.right; 

// Populate right child’s next field if iter is not the last node of this 
// level. 

if (iter.next != null) { 

iter.right.next = iter.next.left; 

} 

iter = iter.next; 

} 

} 


Since we perform (9(1) computation per node, the time complexity is 0(n). The space 
complexity is 0(1). 

Variant: Solve the same problem when there is no level-next field. Your result should 
be stored in the right child field. 

Variant: Solve the same problem for a general binary tree. See Figure 10.7 for an 
example. 
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10.17 Implement locking in a binary tree 


This problem is concerned with the design of an API for setting the state of nodes 
in a binary tree to lock or unlock. A node's state cannot be set to lock if any of its 
descendants or ancestors are in lock. 

Changing a node's state to lock does not change the state of any other nodes. For 
example, all leaves may simultaneously be in state lock. (If this is the case, no nonleaf 
nodes can be in state lock.) 

Write the following methods for a binary tree node class: 

1. A function to test if the node is locked. 

2. A function to lock the node. If the node cannot be locked, return false, otherwise 
lock it and return true. 

3. A function to unlock the node. 

Assume that each node has a parent field. The API will be used in a single 
threaded program, so there is no need for concurrency constructs such as mutexes or 
synchronization. 

Hint: Track the number of locked nodes for each subtree. 

Solution: The brute-force approach is to have a Boolean-valued lock variable for each 
node. Testing if a node is locked is trivial, we simply return that field. Locking is 
more involved—we must visit all the node's ancestors and descendants to see if any 
of them are already locked. If so we cannot lock the node. Unlocking is trivial, we 
simply change the lock variable to false. 

The problem with the brute-force approach is that the time complexity to lock is 
very high— 0(m + d), where m is the number of nodes in the node's subtree, and d is 
the depth of the node. If the node is the root, the time complexity is 0(n), where n is 
the number of nodes. 

The insight to improving the time complexity is that we do not actually care which 
nodes in a node's subtree are locked—all we need to know is whether any node is 
locked or not. We can achieve this with a little extra book-keeping. Specifically, for 
each node we have an additional field which counts the number of nodes in that 
node's subtree that are locked. This makes locking straightforward—to test if any 
descendant is locked, we just look at the count. Testing if an ancestor is locked is 
done as before. If lock succeeds, we have to update the lock counts. The only nodes 
affected are the ones on the path from the root to the given node. Unlocking is slightly 
more involved than before. Specifically, we must reduce the locked node count for 
all ancestors. 

public static class BinaryTree { 

private BinaryTree left, right, parent; 
private boolean locked = false; 
private int numLockedDescendants = ®; 

public boolean isLockedO { return locked; } 

public boolean lockQ { 

// Fife cannot lock if any of this node’s descendants are locked. 
if (numLockedDescendants > ® | | locked) { 
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return false; 


} 

// Fife cannot lock if any of this node’s ancestors are locked. 
for (BinaryTree iter = parent; iter != null; iter = iter.parent) { 
if (iter.locked) { 
return false; 

} 

} 

// Lock this node and increments all its ancestors ’ s descendant lock 
// counts. 
locked = true; 

for (BinaryTree iter = parent; iter != null; iter = iter.parent) { 

++iter.numLockedDescendants; 

} 

return true; 

} 

public void unlock() { 
if (locked) { 

// Unlocks itself and decrements its ancestors’s descendant lock counts. 
locked = false; 

for (BinaryTree iter = parent; iter != null; iter = iter.parent) { 

--iter.numLockedDescendants; 

} 

} 

} 

} 

The time complexity for locking and unlocking is bounded by the depth of the node, 
which, in the worst-case is the tree height, i.e., 0(h). The additional space complexity 
is 0(1) for each node (the count field), i.e., 0(n) for the entire tree. The time complexity 
for checking whether a node is already locked remains 0(1). 
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Chapter 


Heaps 

Using F-heaps we are able to obtain improved running 
times for several network optimization algorithms. 

— "Fibonacci heaps and their uses," 
M. L. Fredman and R. E. Tarjan, 1987 


A heap (also referred to as a priority queue) is a specialized binary tree. Specifically, 
it is a complete binary tree as defined on Page 151. The keys must satisfy the heap 
property —the key at each node is at least as great as the keys stored at its children. See 
Figure 11.1(a) for an example of a max-heap. A max-heap can be implemented as an 
array; the children of the node at index i are at indices 2 i +1 and 2 i + 2. The array rep¬ 
resentation for the max-heap in Figure 11.1(a) is (561,314,401,28,156,359,271,11,3). 

A max-heap supports <9(log n) insertions, 0(1) time lookup for the max element, 
and <9(log n) deletion of the max element. The extract-max operation is defined to 
delete and return the maximum element. See Figure 11.1(b) for an example of deletion 
of the max element. Searching for arbitrary keys has 0(n) time complexity. 

The min-heap is a completely symmetric version of the data structure and supports 
0(1) time lookups for the minimum element. 



(a) A max-heap. Note that the root hold the maxi¬ 
mum key, 561. 



m 


(b) After the deletion of max of the heap in (a). Dele¬ 
tion is performed by replacing the root’s key with the 
key at the last leaf and then recovering the heap prop¬ 
erty by repeatedly exchanging keys with children. 


Figure 11.1: A max-heap and deletion on that max-heap. 


Heaps boot camp 

Suppose you were asked to write a program which takes a sequence of strings pre¬ 
sented in "streaming" fashion: you cannot back up to read an earlier value. Your 
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program must compute the k longest strings in the sequence. All that is required is 
the k longest strings—it is not required to order these strings. 

As we process the input, we want to track the k longest strings seen so far. Out 
of these k strings, the string to be evicted when a longer string is to be added is 
the shortest one. A min-heap (not a max-heap!) is the right data structure for 
this application, since it supports efficient find-min, remove-min, and insert. In the 
program below we use a heap with a custom compare function, wherein strings are 
ordered by length. 


public static List<String> topK(int k, Iterator<String> iter) { 

PriorityQueue<String> minHeap 

= new PriorityQueueo(k, new Comparator<String>() { 
public int compare(String si, String s2) { 

return Integer . compare (si. lengthO , s2 . length() ) ; 

} 

}); 

while (iter.hasNext()) { 
minHeap.add(iter.next()); 
if (minHeap.size() > k) { 

// Remove the shortest string. Note that the comparison function above 
// will order the strings by length. 
minHeap.poll(); 

} 

} 

return new ArrayListo(minHeap) ; 

} 


Each string is processed in O(logk) time, which is the time to add and to remove the 
minimum element from the heap. Therefore, if there are n strings in the input, the 
time complexity to process all of them is 0(n log k). 

We could improve best-case time complexity by first comparing the new string's 
length with the length of the string at the top of the heap (getting this string takes 
0( 1) time) and skipping the insert if the new string is too short to be in the set. 


Use a heap when all you care about is the largest or smallest elements, and you 
do not need to support fast lookup, delete, or search operations for arbitrary 
elements. [Problem 11.1] 

A heap is a good choice when you need to compute the k largest or k smallest 
elements in a collection. For the former, use a min-heap, for the latter, use a 
max-heap. [Problem 11.4] 


Know your heap libraries 

The implementation of a heap in the Java Collections framework is referred to as 
a priority queue; the class is PriorityQueue. The key methods are add(“Gauss”), 
peek(), and poll(), and are straightforward, the latter two returning null when the 
heap is empty. It is possible to specify a custom comparator in the heap constructor, 
as illustrated on on the current page. 
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11.1 Merge sorted files 


This problem is motivated by the following scenario. You are given 500 files, each 
containing stock trade information for an S&P 500 company. Each trade is encoded 
by a line in the following format: 1232111, AAPL ,30,456.12. 

The first number is the time of the trade expressed as the number of milliseconds 
since the start of the day's trading. Lines within each file are sorted in increasing order 
of time. The remaining values are the stock symbol, number of shares, and price. You 
are to create a single file containing all the trades from the 500 files, sorted in order of 
increasing trade times. The individual files are of the order of 5-100 megabytes; the 
combined file will be of the order of five gigabytes. In the abstract, we are trying to 
solve the following problem. 

Write a program that takes as input a set of sorted sequences and computes the union 
of these sequences as a sorted sequence. For example, if the input is (3,5,7), (0,6), 
and (0,6,28), then the output is (0,0,3,5,6,6,7,28). 

Hint: Which part of each sequence is significant as the algorithm executes? 

Solution: A brute-force approach is to concatenate these sequences into a single array 
and then sort it. The time complexity is 0(n log n), assuming there are n elements in 
total. 

The brute-force approach does not use the fact that the individual sequences are 
sorted. We can take advantage of this fact by restricting our attention to the first 
remaining element in each sequence. Specifically, we repeatedly pick the smallest 
element amongst the first element of each of the remaining part of each of the se¬ 
quences. 

A min-heap is ideal for maintaining a collection of elements when we need to add 
arbitrary values and extract the smallest element. 

For ease of exposition, we show how to merge sorted arrays, rather than files. As a 
concrete example, suppose there are three sorted arrays to be merged: (3,5,7), (0,6), 
and (0,6,28). For simplicity, we show the min-heap as containing entries from these 
three arrays. In practice, we need additional information for each entry, namely the 
array it is from, and its index in that array. (In the file case we do not need to explicitly 
maintain an index for next unprocessed element in each sequence—the file I/O library 
tracks the first unread entry in the file.) 

The min-heap is initialized to the first entry of each array, i.e., it is {3,0,0}. We 
extract the smallest entry, 0, and add it to the output which is (0). Then we add 6 to 
the min-heap which is {3,0,6} now. (We chose the 0 entry corresponding to the third 
array arbitrarily, it would be a perfectly acceptable to choose from the second array.) 
Next, extract 0, and add it to the output which is (0,0); then add 6 to the min-heap 
which is {3,6,6}. Next, extract 3, and add it to the output which is (0,0,3); then add 
5 to the min-heap which is {5,6,6}. Next, extract 5, and add it to the output which 
is (0,0,3,5); then add 7 to the min-heap which is {7,6,6}. Next, extract 6, and add 
it to the output which is (0,0,3,5,6); assuming 6 is selected from the second array, 
which has no remaining elements, the min-heap is {7,6}. Next, extract 6, and add it 
to the output which is (0,0,3,5,6,6); then add 28 to the min-heap which is {7,28}. 
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Next, extract 7, and add it to the output which is (0,0,3,5,6,6,7); the min-heap is 
{28}. Next, extract 28, and add it to the output which is (0,0,3,5,6,6,7,28); now, all 
elements are processed and the output stores the sorted elements. 


private static class ArrayEntry { 
public Integer value; 
public Integer arrayld; 

public ArrayEntry(Integer value, Integer arrayld) { 
this.value = value; 
this.arrayld = arrayld; 

} 

} 

public static List<Integer> mergeSortedArrays( 

List<Listdnteger>> sortedArrays) { 

List dterator dnteger >> iters = new ArrayList <>(sortedArrays . size () ) ; 
for (Listdnteger> array : sortedArrays) { 
iters.add(array.iterator ()) ; 

} 

PriorityQueue<ArrayEntry> minHeap = new PriorityQueue<>( 

((int)sortedArrays.size()), new Comparator<ArrayEntry>() { 
©Override 

public int compare(ArrayEntry ol, ArrayEntry o2) { 
return Integer.compare(ol.value, o2.value); 

} 

}); 

for (int i = Q; i < iters. size () ; ++i) { 
if (iters.get(i).hasNext()) { 

minHeap.add(new ArrayEntry(iters.get(i).next(), i)); 

} 

} 

Listdnteger> result = new ArrayList<>() ; 
while (!minHeap.isEmpty()) { 

ArrayEntry headEntry = minHeap.poll(); 
result.add(headEntry.value); 

if (iters.get(headEntry.arrayld).hasNext()) { 

minHeap.add(new ArrayEntry(iters.get(headEntry.arrayld).next(), 

headEntry.arrayld)); 

} 

} 

return result; 


Let k be the number of input sequences. Then there are no more than k elements in 
the min-heap. Both extract-min and insert take <9(log k) time. Hence, we can do the 
merge in 0(n log k) time. The space complexity is 0(k) beyond the space needed to 
write the final result. In particular, if the data comes from files and is written to a file, 
instead of arrays, we would need only 0(k) additional storage. 

Alternatively, we could recursively merge the k files, two at a time using the merge 
step from merge sort. We would go from k to /c/2 then /c/4, etc. files. There would 
be log k stages, and each has time complexity 0(n), so the time complexity is the 
same as that of the heap-based approach, i.e., 0(n log/c). The space complexity of 
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any reasonable implementation of merge sort would end up being 0(n), which is 
considerably worse than the heap based approach when k « n. 

11.2 Sort an increasing-decreasing array 

An array is said to be /c-increasing-decreasing if elements repeatedly increase up to a 
certain index after which they decrease, then again increase, a total of k times. This is 
illustrated in Figure 11.2. 



57 

131 

493 

294 

221 

339 

418 

452 

442 

190 

A[ 0 ] 

A[ 1 ] 

A[ 2 ] 

A[ 3 ] 

A[4] 

A[ 5 ] 

A[ 6 ] 

A[7] 

A[S] 

A[ 9 ] 


Figure 11.2: A 4-increasing-decreasing array. 


Design an efficient algorithm for sorting a /c-increasing-decreasing array. 

Hint: Can you cast this in terms of combining k sorted arrays? 

Solution: The brute-force approach is to sort the array, without taking advantage 
of the /c-increasing-decreasing property. Sorting algorithms run in time 0(n log n), 
where n is the length of the array. 

If k is significantly smaller than n we can do better. For example, if k = 2, the input 
array consists of two subarrays, one increasing, the other decreasing. Reversing the 
second subarray yields two sorted arrays, and the result is their merge. It is fairly 
easy to merge two sorted arrays in 0(n) time. 

Generalizing, we could first reverse the order of each of the decreasing subarrays. 
For the example in Figure 11.2, we would decompose A into four sorted arrays— 
(57,131,493), (221,294), (339,418,452), and (190,442). Now we can use the techniques 
in Solution 11.1 on Page 177 to merge these. 

public static List<Integer> sortKIncreasingDecreasingArray(Listdnteger> A) { 

// Decomposes A into a set of sorted arrays. 

List<List<Integer>> sortedSubarrays = new ArrayList<>(); 

SubarrayType subarrayType = SubarrayType.INCREASING; 
int startldx = Q; 

for (int i = 1; i <= A.sizeO; ++i) { 

if (i == A.sizeO // A is ended. Adds the last subarray 
|| (A.get(i - 1) < A.get(i) 

&<& subarrayType == SubarrayType . DECREASING) 

|| (A.get(i - 1) >= A.get(i) 

&<& subarrayType == SubarrayType . INCREASING)) { 

List dnteger > subList = A.subList(startldx, i) ; 
if (subarrayType == SubarrayType.DECREASING) { 

Collections.reverse(subList); 

} 

sortedSubarrays.add(subList); 
startldx = i; 
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subarrayType = (subarrayType == SubarrayType.INCREASING 
? SubarrayType.DECREASING 
: SubarrayType.INCREASING); 

} 

} 

return MergeSortedArrays.mergeSortedArrays(sortedSubarrays); 

} 

private static enum SubarrayType { INCREASING, DECREASING } 


Just as in Solution 11.1 on Page 177, the time complexity is 0(n log A:) time. 


11.3 Sort AN ALMOST-SORTED ARRAY 

Often data is almost-sorted—for example, a server receives timestamped stock quotes 
and earlier quotes may arrive slightly after later quotes because of differences in server 
loads and network routes. In this problem we address efficient ways to sort such data. 

Write a program which takes as input a very long sequence of numbers and prints 
the numbers in sorted order. Each number is at most k away from its correctly sorted 
position. (Such an array is sometimes referred to as being k-sorted.) For example, no 
number in the sequence (3, -1,2,6,4,5,8) is more than 2 away from its final sorted 
position. 

Hint: How many numbers must you read after reading the ith number to be sure you can place 
it in the correct location? 

Solution: The brute-force approach is to put the sequence in an array, sort it, and 
then print it. The time complexity is 0(n log n), where n is the length of the input 
sequence. The space complexity is 0(n). 

We can do better by taking advantage of the almost-sorted property. Specifically, 
after we have read k + 1 numbers, the smallest number in that group must be smaller 
than all following numbers. For the given example, after we have read the first 3 
numbers, 3,-1,2, the smallest, — 1, must be globally the smallest. This is because the 
sequence was specified to have the property that every number is at most 2 away 
from its final sorted location and the smallest number is at index 0 in sorted order. 
After we read in the 4, the second smallest number must be the minimum of 3,2,4, 
i.e., 2. 

To solve this problem in the general setting, we need to store k + 1 numbers and 
want to be able to efficiently extract the minimum number and add a new number. A 
min-heap is exactly what we need. We add the first k numbers to a min-heap. Now, 
we add additional numbers to the min-heap and extract the minimum from the heap. 
(When the numbers run out, we just perform the extraction.) 

public static void sortApproximatelySortedData(Iterator<Integer> sequence, 

int k) { 

PriorityQueuednteger> minHeap = new PriorityQueue<>(); 

// Adds the first k elements into minHeap. Stop if there are fewer than k 
// elements. 

for (int i = Q; i < k && sequence.hasNext () ; ++i) { 
minHeap.add(sequence.next ()) ; 
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} 


// For every new element, add it to minHeap and extract the smallest. 
while (sequence.hasNext () ) { 
minHeap.add(sequence.next()); 

Integer smallest = minHeap.remove(); 

System.out.println(smallest); 


// sequence is exhausted, iteratively extracts the remaining elements. 
while (!minHeap.isEmpty()) { 

Integer smallest = minHeap.remove(); 

System.out.println(smallest); 

} 

} 


The time complexity is 0(n log k). The space complexity is 0(k). 


11.4 Compute the k closest stars 

Consider a coordinate system for the Milky Way, in which Earth is at (0,0,0). Model 
stars as points, and assume distances are in light years. The Milky Way consists of 
approximately 10 12 stars, and their coordinates are stored in a file. 

How would you compute the k stars which are closest to Earth? 

Hint: Suppose you know the k closest stars in the first n stars. If the (n + l)th star is to be added 
to the set of k closest stars, which element in that set should be evicted? 

Solution: If RAM was not a limitation, we could read the data into an array, and com¬ 
pute the k smallest elements using sorting. Alternatively, we could use Solution 12.8 
on Page 200 to find the kth. smallest element, after which it is easy to find the k smallest 
elements. For both, the space complexity is 0(n), which, for the given dataset, cannot 
be stored in RAM. 

Intuitively, we only care about stars close to Earth. Therefore, we can keep a set 
of candidates, and iteratively update the candidate set. The candidates are the k 
closest stars we have seen so far. When we examine a new star, we want to see if 
it should be added to the candidates. This entails comparing the candidate that is 
furthest from Earth with the new star. To find this candidate efficiently, we should 
store the candidates in a container that supports efficiently extracting the maximum 
and adding a new member. 

A max-heap is perfect for this application. Conceptually, we start by adding the 
first k stars to the max-heap. As we process the stars, each time we encounter a new 
star that is closer to Earth than the star which is the furthest from Earth among the 
stars in the max-heap, we delete from the max-heap, and add the new one. Otherwise, 
we discard the new star and continue. We can simplify the code somewhat by simply 
adding each star to the max-heap, and discarding the maximum element from the 
max-heap once it contains k + 1 elements. 

public static class Star implements Comparable<Star> { 
private double x, y, z; 
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public Star(double x, double y, double z) { 
this.x = x; 
this.y = y; 
this.z = z; 

} 

public double distance() { return Math.sqrt(x *x+y*y+z*z); } 
©Override 

public int compareTo(Star rhs) { 

return Double.compare (this .distance(), rhs.distance()); 

} 

} 

public static List<Star> findClosestKStars(int k, Iterator<Star> stars) { 

// maxHeap to store the closest k stars seen so far. 

PriorityQueue<Star> maxHeap 

= new PriorityQueueo(k, Collections .reverseOrder ()) ; 
while (stars.hasNext()) { 

// Add each star to the max-heap. If the max-heap size exceeds k, remove 
// the maximum element from the max-heap. 

Star star = stars.next(); 
maxHeap.add(star); 
if (maxHeap.size() == k + 1) { 
maxHeap.remove(); 

} 

} 

List<Star> orderedStars = new ArrayList<Star>(maxHeap); 

// The only guarantee PriorityQueue makes about ordering is that the 
// maximum element comes first, so we sort orderedStars. 

Collections.sort(orderedStars); 
return orderedStars; 


The time complexity is 0(n log A:) and the space complexity is 0(k). 

Variant: Design an 0(n log k) time algorithm that reads a sequence of n elements and 
for each element, starting from the kth element, prints the /cth largest element read up 
to that point. The length of the sequence is not known in advance. Your algorithm 
cannot use more than 0(k) additional storage. What are the worst-case inputs for 
your algorithm? 


11.5 Compute the median of online data 

You want to compute the running median of a sequence of numbers. The sequence is 
presented to you in a streaming fashion—you cannot back up to read an earlier value, 
and you need to output the median after reading in each new element. For example, 
if the input is 1,0,3,5,2,0,1 the output is 1,0.5,1,2,2,1.5,1. 

Design an algorithm for computing the running median of a sequence. 

Hint: Avoid looking at all values each time you read a new value. 
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Solution: The brute-force approach is to store all the elements seen so far in an array 
and compute the median using, for example Solution 12.8 on Page 200 for finding 
the kth smallest entry in an array. This has time complexity 0(n 2 ) for computing the 
running median for the first n elements. 

The shortcoming of the brute-force approach is that it is not incremental, i.e., it 
does not take advantage of the previous computation. Note that the median of a 
collection divides the collection into two equal parts. When a new element is added 
to the collection, the parts can change by at most one element, and the element to be 
moved is the largest of the smaller half or the smallest of the larger half. 

We can use two heaps, a max-heap for the smaller half and a min-heap for the 
larger half. We will keep these heaps balanced in size. The max-heap has the property 
that we can efficiently extract the largest element in the smaller part; the min-heap is 
similar. 

For example, let the input values be 1,0,3,5,2,0,1. Let L and H be the contents of 
the min-heap and the max-heap, respectively. Here is how they progress: 

1. Read in 1: L = [1],H = [], median is 1. 

2. Read in 0: L = [1],H = [0], median is (1 + 0)/2 = 0.5. 

3. Read in 3: L = [1,3],H = [0], median is 1. 

4. Read in 5: L = [3,5],H = [1,0], median is (3 +1)/2 = 2. 

5. Read in 2: L = [2,3,5],H = [1,0], median is 2. 

6. Read in 0: L = [2,3,5], H = [1,0,0], median is (2 + l)/2 = 1.5. 

7. Read in 1: L = [1,2,3,5], H = [1,0,0], median is 1. 

private static final int DEFAULT_INITIAL_CAPACITY = 16; 

public static void onlineMedian(Iterator<Integer> sequence) { 

// minHeap stores the larger half seen so far. 

PriorityQueue<Integer> minHeap = new PriorityQueue<>(); 

// maxHeap stores the smaller half seen so far. 

PriorityQueue<Integer> maxHeap = new PriorityQueue<>( 

DEFAULT_INITIAL_CAPACITY, Collections.reverseOrder()); 

while (sequence.hasNext()) { 
int x = sequence . next () ; 
if (minHeap.isEmpty()) { 

// This is the very first element. 
minHeap.add(x); 

} else { 

if (x >= minHeap.peek()) { 
minHeap.add(x) ; 

} else { 

maxHeap.add(x); 

} 

} 

// Ensure minHeap and maxHeap have equal number of elements if 
// an even number of elements is read; otherwise, minHeap must have 
// one more element than maxHeap. 
if (minHeap.size() > maxHeap.size() + 1) { 
maxHeap.add(minHeap.remove()); 

} else if (maxHeap.size() > minHeap.size()) { 
minHeap.add(maxHeap.remove()); 

} 
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System.out.println(minHeap.size() == maxHeap.size() 

? 0.5 * (minHeap.peek() + maxHeap.peek()) 
: minHeap .peekQ) ; 

} 

} 


The time complexity per entry is <9(log n), corresponding to insertion and extraction 
from a heap. 


11.6 Compute the k largest elements in a max-heap 

A heap contains limited information about the ordering of elements, so unlike a sorted 
array or a balanced BST, naive algorithms for computing the k largest elements have 
a time complexity that depends linearly on the number of elements in the collection. 

Given a max-heap, represented as an array A, design an algorithm that computes the k 
largest elements stored in the max-heap. You cannot modify the heap. For example, if 
the heap is the one shown in Figure 11.1(a) on Page 175, then the array representation 
is (561,314,401,28,156,359,271,11,3), the four largest elements are 561,314,401, and 
359. 

Solution: The brute-force algorithm is to perform k extract-max operations. The time 
complexity is O(k\ogn), where n is the number of elements in the heap. Note that 
this algorithm entails modifying the heap. 

Another approach is to use an algorithm for finding the kth smallest element in 
an array, such as the one described in Solution 12.8 on Page 200. That has time 
complexity almost certain <9(n), and it too modifies the heap. 

The following algorithm is based on the insight that the heap has partial order 
information, specifically, a parent node always stores value greater than or equal to 
the values stored at its children. Therefore, the root, which is stored in A[0], must 
be one of the k largest elements—in fact, it is the largest element. The second largest 
element must be the larger of the root's children, which are A[l] and A[ 2]—this is the 
index we continue processing from. 

The ideal data structure for tracking the index to process next is a data structure 
which support fast insertions, and fast extract-max, i.e., in a max-heap. So our 
algorithm is to create a max-heap of candidates, initialized to hold the index 0, which 
serves as a reference to A[ 0]. The indices in the max-heap are ordered according to 
corresponding value in A. We then iteratively perform k extract-max operations from 
the max-heap. Each extraction of an index i is followed by inserting the indices of 
i's left child, 2i + 1, and right child, 2i + 2, to the max-heap, assuming these children 
exist. 

private static class HeapEntry { 
public Integer index; 
public Integer value; 

public HeapEntry(Integer index, Integer value) { 
this. index = index; 
this. value = value; 
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} 

} 

private static class Compare implements Comparator<HeapEntry> { 

©Override 

public int compare(HeapEntry ol, HeapEntry o2) { 
return Integer.compare(o2.value, ol.value); 

} 

public static final Compare COMPARE_HEAP_ENTRIES = new Compare(); 

} 

private static final int DEFAULT_INITIAL_CAPACITY = 16; 

public static List<Integer> kLargestlnBinaryHeap(Listdnteger> A, int k) { 
if (k <= ®) { 

return Collections.EMPTY_LIST; 

} 

// Stores the (index, value)-pair in candidateMaxHeap. This heap is 
// ordered by the value field. 

PriorityQueuecHeapEntry> candidateMaxHeap = new PriorityQueue<>( 
DEFAULT_INITIAL_CAPACITY, Compare.COMPARE_HEAP_ENTRIES); 
candidateMaxHeap.add(new HeapEntry(®, A.get(®))); 

List<Integer> result = new ArrayList<>(); 
for (int i = ®; i < k; ++i) { 

Integer candidateldx = candidateMaxHeap.peek().index; 
result.add(candidateMaxHeap.remove().value); 

Integer leftChildldx = 2 * candidateldx + 1; 
if (leftChildldx < A.sizeO) { 

candidateMaxHeap.add(new HeapEntry(leftChildldx, A.get(leftChildldx))); 

> 

Integer rightChildldx = 2 * candidateldx + 2; 
if (rightChildldx < A.sizeO) { 
candidateMaxHeap.add( 

new HeapEntry(rightChildldx, A.get(rightChildldx))); 

} 

} 

return result; 


The total number of insertion and extract-max operations is 0(k ), yielding an 0(k log k) 
time complexity, and an 0(k) additional space complexity. This algorithm does not 
modify the original heap. 


11.7 Implement a stack API using a heap 

We discussed the notion of reduction when describing problem solving patterns. 
Usually, reductions are used to solve a more complex problem using a solution to a 
simpler problem as a subroutine. 

Occasionally it makes sense to go the other way—for example, if we need the 
functionality of a heap, we can use a BST library, which is more commonly available. 
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with modest performance penalties with respect, for example, to an array-based 
implementation of a heap. 

How would you implement a stack API using a heap? 

Hint: Store an additional value with each element that is inserted. 

Solution: The key property of a stack is that elements are removed in LIFO order. 
Since we need to implement this property using a heap, we should look at ways of 
tracking the insertion order using the heap property. 

We can use a global "timestamp" for each element, which we increment on each 
insert. We use this timestamp to order elements in a max-heap. This way the most 
recently added element is at the root, which is exactly what we want. 

private static final int DEFAULT_INITIAL_CAPACITY = 16; 

private static class ValueWithRank { 
public Integer value; 
public Integer rank; 

public ValueWithRank(Integer value, Integer rank) { 
this. value = value; 
this. rank = rank; 

} 

} 

private static class Compare implements Comparator<ValueWithRank> { 

©Override 

public int compare(ValueWithRank ol, ValueWithRank o2) { 
return Integer.compare(o2.rank, ol.rank); 

} 

public static final Compare COMPARE.VALUEWITHRANK = new Compare(); 

} 

public static class Stack { 
private int timestamp = 0; 

private PriorityQueue<ValueWithRank> maxHeap = new PriorityQueue<>( 
DEFAULT_INITIAL_CAPACITY, Compare.COMPARE_VALUEWITHRANK); 

public void push(Integer x) { 

maxHeap. add(new ValueWithRank(x, timestamp++)); 

} 

public Integer pop() throws NoSuchElementException { 
return maxHeap.remove().value; 

} 

public Integer peek() { return maxHeap.peek().value; } 

} 


The time complexity for push and pop is that of adding and extracting-max from a 
max-heap, i.e., <9(log n), where n is the current number of elements. 

Variant: How would you implement a queue API using a heap? 
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Chapter 


Searching 



— "The Anatomy of A Large-Scale Hypertextual Web Search Engine," 

S. M. Brin and L. Page, 1998 

Search algorithms can be classified in a number of ways. Is the underlying collection 
static or dynamic, i.e., inserts and deletes are interleaved with searching? Is it worth 
spending the computational cost to preprocess the data so as to speed up subsequent 
queries? Are there statistical properties of the data that can be exploited? Should we 
operate directly on the data or transform it? 

In this chapter, our focus is on static data stored in sorted order in an array Data 
structures appropriate for dynamic updates are the subject of Chapters 11,13, and 15. 

The first collection of problems in this chapter are related to binary search. The 
second collection pertains to general search. 

Binary search 

Given an arbitrary collection of n keys, the only way to determine if a search key is 
present is by examining each element. This has 0(n) time complexity. Fundamentally, 
binary search is a natural elimination-based strategy for searching a sorted array. The 
idea is to eliminate half the keys from consideration by keeping the keys in sorted 


187 





order. If the search key is not equal to the middle element of the array, one of the two 
sets of keys to the left and to the right of the middle element can be eliminated from 
further consideration. 

Questions based on binary search are ideal from the interviewers perspective: it 
is a basic technique that every reasonable candidate is supposed to know and it can 
be implemented in a few lines of code. On the other hand, binary search is much 
trickier to implement correctly than it appears—you should implement it as well as 
write corner case tests to ensure you understand it properly. 

Many published implementations are incorrect in subtle and not-so-subtle ways— 
a study reported that it is correctly implemented in only five out of twenty textbooks. 
Jon Bentley, in his book " Programming Pearls'' reported that he assigned binary search 
in a course for professional programmers and found that 90% failed to code it correctly 
despite having ample time. (Bentley's students would have been gratified to know 
that his own published implementation of binary search, in a column titled "Writing 
Correct Programs", contained a bug that remained undetected for over twenty years.) 

Binary search can be written in many ways—recursive, iterative, different idioms 
for conditionals, etc. Here is an iterative implementation adapted from Bentley's 
book, which includes his bug. 

public static int bsearch(int t, ArrayList<Integer> A) { 
int L = ®, U = A.sizeC) - 1; 
while (L <= U) { 

int M = (L + U) / 2; 
if (A.get(M) < t) { 

L = M + 1; 

} else if (A.get(M) == t) { 
return M; 

} else { 

U = M - 1; 

} 

} 

return -1; 

} 


The error is in the assignment M=(L + U)/2in Line 4, which can potentially 
lead to overflow. This overflow can be avoided by using M = L + (U-L)/2. 

The time complexity of binary search is given by T(n) = T(n/2) + c, where c is a 
constant. This solves to T(n) = <9(logn), which is far superior to the 0(n) approach 
needed when the keys are unsorted. A disadvantage of binary search is that it requires 
a sorted array and sorting an array takes 0(n log n) time. However, if there are many 
searches to perform, the time taken to sort is not an issue. 

Many variants of searching a sorted array require a little more thinking and create 
opportunities for missing corner cases. 

Searching boot camp 

When objects are comparable, they can be sorted and searched for using library 
search functions. Typically, the language knows how to compare built-in types, e.g., 
integers, strings, library classes for date, URLs, SQL timestamps, etc. However, user- 
defined types used in sorted collections must explicitly implement comparision, and 
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ensure this comparision has basic properties such as transitivity. (If the comparision 
is implemented incorrectly, you may find a lookup into a sorted collection fails, even 
when the item is present.) 

Suppose we are given as input an array of students, sorted by descending GPA, 
with ties broken on name. In the program below, we show how to use the library 
binary search routine to perform fast searches in this array. In particular, we pass 
binary search a custom comparator which compares students on GPA (higher GPA 
comes first), with ties broken on name. 

public static class Student { 
public String name; 
public double gradePointAverage; 

Student(String name, double gradePointAverage) { 
this.name = name; 

this .gradePointAverage = gradePointAverage; 

} 

} 

private static final Comparator<Student> compGPA = new Comparator<Student>() { 
©Override 

public int compare(Student a, Student b) { 

if (a.gradePointAverage != b.gradePointAverage) { 

return Double.compare(a.gradePointAverage, b.gradePointAverage); 

} 

return a.name.compareTo(b.name); 

} 

}; 


public static boolean searchStudent(List<Student> students, Student target, 

Comparator<Student> compGPA) { 
return Collections.binarySearch(students, target, compGPA) >= 0; 

} 


Assuming the z-th element in the sequence can be accessed in (9(1) time, the time 
complexity of the program is <9(log n). 


Binary search is an effective search tool. It is applicable to more than just searching 
in sorted arrays, e.g., it can be used to search an interval of real numbers or 
integers. [Problem 12.4] 

If your solution uses sorting, and the computation performed after sorting is 
faster than sorting, e.g., 0(n) or (9(logn), look for solutions that do not perform 
a complete sort. [Problem 12.8] 

Consider time/space tradeoffs, such as making multiple passes through the data. 
[Problem 12.9] 


Know your searching libraries 

Searching is a very broad concept, and it is present in many data structures. For 
example contains(e) is present in ArrayList, LinkedList, HashSet, and TreeSet, 
albeit with very different time complexities. Here we focus on binary search. 


189 



• To search an array, use Arrays. binary Search (A, “Euler”) • The time complex¬ 
ity is <9(log n), where n is length of the array. 

• To search a sorted Li st-type object, use Collections, binary Sear ch(li st, 42). 
These return the index of the searched key if it is present, and a negative value 
if it is not present. 

• The time complexity depends on the nature of the List implementation. For 
ArrayList (and more generally, any List implementation in which positional 
access is constant-time), it is <9(log n), where n is the number of elements in the 
list. For LinkedList it is 0(n). 

• When there are multiple occurrences of the search key, neither Arrays nor 
Collections offer any guarantees as to which one will be found by binary 
search. 

• If the search key is not present, both methods return (-(insertion point) - 1), 
where insertion point is defined as the point at which the key would be inserted 
into the array, i.e., the index of the first element greater than the key, or the 
number of elements if all elements are less than the specified value. 

12.1 Search a sorted array for first occurrence of k 

Binary search commonly asks for the index of any element of a sorted array that is 
equal to a specified element. The following problem has a slight twist on this. 
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Figure 12.1: A sorted array with repeated elements. 


Write a method that takes a sorted array and a key and returns the index of the 
first occurrence of that key in the array. For example, when applied to the array in 
Figure 12.1 your algorithm should return 3 if the given key is 108; if it is 285, your 
algorithm should return 6. 

Hint: What happens when every entry equals k ? Don't stop when you first see k. 

Solution: A naive approach is to use binary search to find the index of any element 
equal to the key, k. (If k is not present, we simply return -1.) After finding such an 
element, we traverse backwards from it to find the first occurrence of that element. 
The binary search takes time <9(log n), where n is the number of entries in the array. 
Traversing backwards takes 0(n) time in the worst-case—consider the case where 
entries are equal to k. 

The fundamental idea of binary search is to maintain a set of candidate solutions. 
For the current problem, if we see the element at index i equals k, although we do 
not know whether i is the first element equal to k, we do know that no subsequent 
elements can be the first one. Therefore we remove all elements with index i + 1 or 
more from the candidates. 
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Let's apply the above logic to the given example, with k = 108. We start with all 
indices as candidates, i.e., with [0,9]. The midpoint index, 4 contains k. Therefore we 
can now update the candidate set to [0,3], and record 4 as an occurrence of k. The 
next midpoint is 1, and this index contains -10. We update the candidate set to [2,3]. 
The value at the midpoint 2 is 2, so we update the candidate set to [3,3]. Since the 
value at this midpoint is 108, we update the first seen occurrence of A: to 3. Now the 
interval is [3,2], which is empty, terminating the search—the result is 3. 

public static int searchFirstOfK(List<Integer> A, int k) { 
int left = ®, right = A.sizeO - 1» result = -1; 

// A . subList(left, right + 1) is the candidate set. 
while (left <= right) { 

int mid = left + ((right - left) / 2); 
if (A.get(mid) > k) { 
right = mid - 1; 

} else if (A.get(mid) == k) { 
result = mid; 

// Nothing to the right of mid can be the first occurrence of k. 
right = mid - 1; 

} else { // A. get (mid) < k 
left = mid + 1; 

} 

} 

return result; 

} 


The complexity bound is still <9(log n )—this is because each iteration reduces the size 
of the candidate set by half. 

Variant: Design an efficient algorithm that takes a sorted array and a key, and finds 
the index of the first occurrence of an element greater than that key. For example, 
when applied to the array in Figure 12.1 on the preceding page your algorithm should 
return 9 if the key is 285; if it is -13, your algorithm should return 1. 

Variant: Let A be an unsorted array of n integers, with A[0] > A[l] and A[n - 2] < 
A[n - 1]. Call an index i a local minimum if A[i] is less than or equal to its neighbors. 
How would you efficiently find a local minimum, if one exists? 

Variant: Write a program which takes a sorted array A of integers, and an integer k, 
and returns the interval enclosing k, i.e., the pair of integers L and U such that L is the 
first occurrence of kin A and U is the last occurrence of kin A. If k does not appear 
in A, return [-1,-1]. For example if A = (1,2,2,4,4,4,7,11,11,13) and k = 11, you 
should return [7,8]. 

Variant: Write a program which tests if p is a prefix of a string in an array of sorted 
strings. 


12.2 Search a sorted array for entry equal to its index 

Design an efficient algorithm that takes a sorted array of distinct integers, and returns 
an index i such that the element at index i equals i. For example, when the input is 
(-2,0,2,3,6,7,9) your algorithm should return 2 or 3. 
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Hint: Reduce this problem to ordinary binary search. 


Solution: A brute-force approach is to iterate through the array, testing whether the 
zth entry equals i. The time complexity is 0(n), where n is the length of the array. 

The brute-force approach does not take advantage of the fact that the array (call 
it A) is sorted and consists of distinct elements. In particular, note that the difference 
between an entry and its index increases by at least 1 as we iterate through A. Observe 
that if A[j] > j, then no entry after j can satisfy the given criterion. This is because 
each element in the array is at least 1 greater than the previous element. For the same 
reason, if A[j] < j, no entry before j can satisfy the given criterion. 

The above observations can be directly used to create a binary search type algo¬ 
rithm for finding an i such that A[i] = i. A slightly simpler approach is to search 
the secondary array B whose z’th entry is A[i] - i for 0, which is just ordinary binary 
search. We do not need to actually create the secondary array, we can simply use 
A[i\ - i wherever B[i] is referenced. 

For the given example, the secondary array B is (-2, —1,0,0,2,2,3). Binary search 
for 0 returns the desired result, i.e., either of index 2 or 3. 

public static int searchEntryEqualToItsIndex(Listdnteger> A) { 
int left = ®, right = A.sizeO - 1; 
while (left <= right) { 

int mid = left + ((right - left) / 2); 
int difference = A.get(mid) - mid; 

// A . get(mid) == mid if and only if difference == ®. 
if (difference == ®) { 
return mid; 

} else if (difference > ®) { 
right = mid - 1; 

} else { // difference < <9. 
left = mid + 1; 

} 

} 

return -1; 

} 


The time complexity is the same as that for binary search , i.e., <9(log n), where n is 
the length of A. 

Variant: Solve the same problem when A is sorted but may contain duplicates. 


12.3 Search a cyclically sorted array 

An array is said to be cyclically sorted if it is possible to cyclically shift its entries so 
that it becomes sorted. For example, the array in Figure 12.2 on the facing page is 
cyclically sorted—a cyclic left shift by 4 leads to a sorted array. 

Design an <9(log n) algorithm for finding the position of the smallest element in a 
cyclically sorted array. Assume all elements are distinct. For example, for the array 
in Figure 12.2 on the next page, your algorithm should return 4. 

Hint: Use the divide and conquer principle. 
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Figure 12.2: A cyclically sorted array. 


Solution: A brute-force approach is to iterate through the array, comparing the 
running minimum with the current entry. The time complexity is 0(n), where n is the 
length of the array 

The brute-force approach does not take advantage of the special properties of the 
array, A. For example, for any m, if A[m] > A[n - 1], then the minimum value must 
be an index in the range [m + 1, n — 1]. Conversely, if A[m\ < A[n - 1], then no index 
in the range [m + \,n - 1] can be the index of the minimum value. (The minimum 
value may be at A[m\.) Note that it is not possible for A[m] = A[n - 1], since it is given 
that all elements are distinct. These two observations are the basis for a binary search 
algorithm, described below. 

public static int searchSmallest(Listdnteger> A) { 
int left = ©, right = A.sizeO " 1; 
while (left < right) { 

int mid = left + ((right - left) / 2); 
if (A.get(mid) > A.get(right)) { 

// Minimum must be in A.subList(mid + 1, right + 1). 
left = mid + 1; 

} else { // A . get(mid) < A.get(right). 

// Minimum cannot be in A.subList(mid + 1, right + 1) so it must be in 
// A . sublist(left, mid + 1). 
right = mid; 

> 

} 

// Loop ends when left == right. 
return left; 

} 


The time complexity is the same as that of binary search, namely <9(log n). 

Note that this problem cannot, in general, be solved in less than linear time when 
elements may be repeated. For example, if A consists of n - 1 Is and a single 0, that 0 
cannot be detected in the worst-case without inspecting every element. 

Variant: A sequence is strictly ascending if each element is greater than its prede¬ 
cessor. Suppose it is known that an array A consists of a strictly ascending sequence 
followed by a strictly a strictly descending sequence. Design an algorithm for finding 
the maximum element in A. 

Variant: Design an (9(log n) algorithm for finding the position of an element k in a 
cyclically sorted array of distinct elements. 
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12.4 Compute the integer square root 


Write a program which takes a nonnegative integer and returns the largest integer 
whose square is less than or equal to the given integer. For example, if the input is 
16, return 4; if the input is 300, return 17, since 17 2 = 289 < 300 and 18 2 = 324 > 300. 

Hint: Look out for a corner-case. 

Solution: A brute-force approach is to square each number from 1 to the key, k, 
stopping as soon as we exceed k. The time complexity is 0(k). For a 64 bit integer, at 
one nanosecond per iteration, this algorithm will take over 500 years. 

Looking more carefully at the problem, it should be clear that it is wasteful to take 
unit-sized increments. For example, if x 2 < k, then no number smaller than x can be 
the result, and if x 2 > k, then no number greater than or equal to x can be the result. 

This ability to eliminate large sets of possibilities is suggestive of binary search. 
Specifically, we can maintain an interval consisting of values whose squares are 
unclassified with respect to k, i.e., might be less than or greater than k. 

We initialize the interval to [0, k\. We compare the square of m = |_(/ + r)/2J with k, 
and use the elimination rule to update the interval. If m 2 < k, we know all integers 
less than or equal to m have a square less than or equal to k. Therefore, we update the 
interval to [m +1, r]. If m 2 > k, we know all numbers greater than or equal to m have a 
square greater than k, so we update the candidate interval to [/, m - 1]. The algorithm 
terminates when the interval is empty, in which case every number less than / has a 
square less than or equal to k and Vs square is greater than k, so the result is Z — 1. 

For example, if k = 21, we initialize the interval to [0,21]. The midpoint m = [(0 + 
21)/2J = 10; since 10 2 > 21, we update the interval to [0,9]. Now m = L(0 + 9)/2J = 4; 
since 4 2 < 21, we update the interval to [5,9]. Now m = |_(5 + 8)/2J = 7; since 7 2 > 21, 
we update the interval to [5,6]. Now m - [(5 + 6)/2J = 5; since 5 2 > 21, we update 
the interval to [5,4]. Now the right endpoint is less than the left endpoint, i.e., the 
interval is empty, so the result is 5 - 1 = 4, which is the value returned. 

For k = 25, the sequence of intervals is [0,25], [0,11], [6,11], [6,7], [6,5]. The re¬ 
turned value is 6 - 1 =5. 

public static int squareRoot (int k) { 
long left = Q, right = k; 

// Candidate interval [left, right] where everything before left has 
// square <= k, and everything after right has square > k. 
while (left <= right) { 

long mid = left + ((right - left) / 2); 
long midSquared = mid * mid; 
if (midSquared <= k) { 
left = mid + 1; 

} else { 

right = mid - 1; 

} 

} 

return (int)left - 1; 

} 


The time complexity is that of binary search over the interval [0 ,k], i.e., <9(logk). 
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12.5 Compute the real square root 

Square root computations can be implemented using sophisticated numerical tech¬ 
niques involving iterative methods and logarithms. However, if you were asked to 
implement a square root function, you would not be expected to know these tech¬ 
niques. 

Implement a function which takes as input a floating point value and returns its 
square root. 

Hint: Iteratively compute a sequence of intervals, each contained in the previous interval, that 
contain the result. 

Solution: Let x be the input. One approach is to find an integer n such that n 2 < x and 
(n + l) 2 > x, using, for example, the approach in Solution 12.4 on the facing page. We 
can then search within [n, n + 1] to find the square root of x to any specified tolerance. 

We can avoid decomposing the computation into an integer computation followed 
by a floating point computation by directly performing binary search. The reason is 
that if a number is too big to be the square root of x, then any number bigger than that 
number can be eliminated. Similarly, if a number is too small to be the square root of 
x, then any number smaller than that number can be eliminated. 

Trivial choices for the initial lower bound and upper bound are 0 and the largest 
floating point number that is representable. The problem with this is that it does not 
play well with finite precision arithmetic—the first midpoint itself will overflow on 
squaring. 

We cannot start with [0,x] because the square root may be larger than x, e.g., 
Vl/4 = 1/2. However, if x > 1.0, we can tighten the lower and upper bounds to 1.0 
and x, respectively, since if 1.0 < x then x < x 2 . On the other hand, if x < 1.0, we 
can use x and 1.0 as the lower and upper bounds respectively, since then the square 
root of x is greater than x but less than 1.0. Note that the floating point square root 
problem differs in a fundamental way from the integer square root (Problem 12.4 on 
the preceding page). In that problem, the initial interval containing the solution is 
always [0,x]. 

public static double squareRoot(double x) { 

// Decides the search range according to x’s value relative to 1.9. 
double left, right; 
if (x < 1.®) { 
left = x; 
right = 1.®; 

} else {// x >= 1.9. 
left = 1.®; 
right = x; 

} 

// Keeps searching as long as left < right, within tolerance. 
while (compare(left, right) == Ordering.SMALLER) { 
double mid = left + ®.5 * (right - left); 
double midSquared = mid * mid; 

if (compare(midSquared, x) == Ordering.EQUAL) { 
return mid; 
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} else if (compare(midSquared, x) == Ordering.LARGER) { 
right = mid; 

} else { 

left = mid; 

} 

} 

return left; 


private static enum Ordering { SMALLER, EQUAL, LARGER } 

private static Ordering compare (double a, double b) { 
final double EPSILON = ®.®®®®1; 

// Uses normalization for precision problem. 
double diff = (a - b) / b; 
return diff < -EPSILON 
? Ordering.SMALLER 

: (diff > EPSILON ? Ordering.LARGER : Ordering.EQUAL); 


The time complexity is <9(log *), where s is the tolerance. 

Variant: Given two positive floating point numbers x and y, how would you compute 
* to within a specified tolerance e if the division operator cannot be used? You 
cannot use any library functions, such as log and exp; addition and multiplication are 
acceptable. 

Generalized search 

Now we consider a number of search problems that do not use the binary search 
principle. For example, they focus on tradeoffs between RAM and computation 
time, avoid wasted comparisons when searching for the minimum and maximum 
simultaneously, use randomization to perform elimination efficiently, use bit-level 
manipulations to identify missing elements, etc. 

12.6 Search in a 2D sorted array 

Call a 2D array sorted if its rows and its columns are nondecreasing. See Figure 12.3 
on the facing page for an example of a 2D sorted array. 

Design an algorithm that takes a 2D sorted array and a number and checks whether 
that number appears in the array. For example, if the input is the 2D sorted array in 
Figure 12.3 on the next page, and the number is 7, your algorithm should return false; 
if the number is 8, your algorithm should return true. 

Hint: Can you eliminate a row or a column per comparison? 

Solution: Let the 2D array be A and the input number be x. We can perform binary 
search on each row independently, which has a time complexity 0(m log n), where m 
is the number of rows and n is the number of columns. (If searching on columns, the 
time complexity is 0(n log m).) 

Note that the above approach fails to take advantage of the fact that both rows 
and columns are sorted—it treats separate rows independently of each other. For 
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example, if x < A[0][0] then no row or column can contain x —the sortedness property 
guarantees that A[0][0] is the smallest element in A. 

However, if x > A[0][0], we cannot eliminate the first row or the first column of 
A. Searching along both rows and columns will lead to a 0(mn) solution, which is far 
worse than the previous solution. The same problem arises if x < A[m - l][n - 1]. 

A good rule of design is to look at extremal cases. We have already seen that there 
is nothing to be gained by comparing with A [0] [0] and A [m -1 ] [n -1 ]. However, there 
are some more extremal cases. For example, suppose we compare x with A[0][rc - 1]. 
If x = A[0][n - 1], we have found the desired value. Otherwise: 

• x > A[0][n - 1], in which case x is greater than all elements in Row 0. 

• x<A[Q][n-l],in which case x is less than all elements in Column n - 1. 

In either case, we have a 2D array with one fewer row or column to search. The other 
extremal case, namely comparing with A[m - 1][0] yields a very similar algorithm. 

For the example in Figure 12.3, if the number is 7, our algorithm compares the 
top-right entry, A[0][4] = 6 with 7. Since 7 > 6, we know 7 cannot be present in 
Row 0. Now we compare with A[l][4] = 21. Since 7 < 21, we know 7 cannot be 
present in Column 4. Now we compare with A[l][3] = 9. Since 7 < 9, we know 7 
cannot be present in Column 3. Now we compare with A[l][2] = 5. Since 7 > 5, 
we know 7 cannot be present in Row 1. Now we compare with A[2] [2] = 6. Since 
7 > 6, we know 7 cannot be present in Row 2. Now we compare with A[3] [2] = 8. 
Since 7 < 8, we know 7 cannot be present in Column 2. Now we compare with 
A[3][l] = 6. Since 7 > 6, we know 7 cannot be present in Row 3. Now we compare 
with A[4][l] = 8. Since 7 < 8, we know 7 cannot be present in Column 1. Now we 
compare with A[4][0] = 6. Since 7 > 6, we know 7 cannot be present in Row 4. Now 
we compare with A[5][0] = 8. Since 7 < 8, we know 7 cannot be present in Column 0. 
There are no remaining entries, so we return false. 

Now suppose we want to see if 8 is present. Proceeding similarly, we eliminate 
Row 0, then Column 4, then Row 1, then Column 3, then Row 2. When we compare 
with A[3][2] we have a match so we return true. 

public static boolean matrixSearch(List<List<Integer» A, int x) { 

int row = Q, col = A.get (0) . size () - 1; // Start from the top-right corner. 

// Keeps searching while there are unclassified rows and columns. 
while (row < A. size Q && col >= ®) { 


197 




if (A.get(row).get(col).equals(x)) { 

return true; 

} else if (A.get(row).get(col) < x) { 
++row; // Eliminate this row. 

} else { // A.get(row).get(col) > x. 
--col; // Eliminate this column. 

} 

} 

return false; 


In each iteration, we remove a row or a column, which means we inspect at most 
m + n - 1 elements, yielding an 0(m + n) time complexity. 


12.7 Find the min and max simultaneously 

Given an array of comparable objects, you can find either the min or the max of the 
elements in the array with n — 1 comparisons, where n is the length of the array. 

Comparing elements may be expensive, e.g., a comparison may involve a number 
of nested calls or the elements being compared may be long strings. Therefore, it is 
natural to ask if both the min and the max can be computed with less than the 2 (n - 1) 
comparisons required to compute the min and the max independently. 

Design an algorithm to find the min and max elements in an array. For example, if 
A = (3,2,5,1,2,4), you should return 1 for the min and 5 for the max. 

Hint: Use the fact that a < b and b < c implies a < c to reduce the number of compares used by 
the brute-force approach. 

Solution: The brute-force approach is to compute the min and the max independently, 
i.e., with 2(n - 1) comparisons. We can reduce the number of comparisons by 1 by 
first computing the min and then skipping the comparison with it when computing 
the max. 

One way to think of this problem is that we are searching for the strongest and 
weakest players in a group of players, assuming players are totally ordered. There is 
no point in looking at any player who won a game when we want to find the weakest 
player. The better approach is to play n /2 matches between disjoint pairs of players. 
The strongest player will come from the n /2 winners and the weakest player will 
come from the n/2 losers. 

Following the above analogy, we partition the array into min candidates and max 
candidates by comparing successive pairs—this will give us n/2 candidates for min 
and n/2 candidates for max at the cost of n/2 comparisons. It takes n/2-1 comparisons 
to find the min from the min candidates and n/2 - 1 comparisons to find the max 
from the max candidates, yielding a total of 3n/2 - 2 comparisons. 

Naively implemented, the above algorithm need 0(n) storage. However, we can 
implement it in streaming fashion, by maintaining candidate min and max as we 
process successive pairs. Note that this entails three comparisons for each pair. 

For the given example, we begin by comparing 3 and 2. Since 3 > 2, we set min to 
2 and max to 3. Next we compare 5 and 1. Since 5 > 1, we compare 5 with the current 
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max, namely 3, and update max to 5. We compare 1 with the current min, namely 2, 
and update min to 1. Then we compare 2 and 4. Since 4 > 2, we compare 4 with the 
current max, namely 5. Since 4 < 5, we do not update max. We compare 2 with the 
current min, namely 1 Since 2 > 1, we do not update min. 

private static class MinMax { 
public Integer min; 
public Integer max; 

public MinMax(Integer min, Integer max) { 
this. min = min; 
this .max = max; 

} 

private static MinMax minMax(Integer a, Integer b) { 

return Integer. compare (b, a) < ® ? new MinMax (b, a) : new MinMax (a, b) ; 

} 


public static MinMax findMinMax(Listdnteger> A) { 

if (A.sizeO <= 1) { 

return new MinMax(A.get(©), A.get(®)); 

} 

MinMax globalMinMax = MinMax.minMax(A.get(®), A.get(l)); 

// Process two elements at a time. 

for (int i = 2; i + 1 < A.sizeO; i += 2) { 

MinMax localMinMax = MinMax.minMax(A.get(i), A.get(i + 1)); 
globalMinMax = new MinMax(Math.min(globalMinMax.min, localMinMax.min), 

Math.max(globalMinMax.max, localMinMax.max)); 

> 

// If there is odd number of elements in the array, we still 

// need to compare the last element with the existing answer. 

if ((A.sizeO % 2) != ®) { 
globalMinMax 

= new MinMax(Math.min(globalMinMax.min, A.get(A.size() - 1)), 

Math.max(globalMinMax.max, A.get(A.size() - 1))); 

} 

return globalMinMax; 


The time complexity is 0(n) and the space complexity is 0{ 1). 

Variant: What is the least number of comparisons required to find the min and the 
max in the worst-case? 


12.8 Find the kru largest element 

Many algorithms require as a subroutine the computation of the kth largest element 
of an array. The first largest element is simply the largest element. The nth largest 
element is the smallest element, where n is the length of the array. 

For example, if the input array A = (3,2,1,5,4), then A [3] is the first largest element 
in A, A[0] is the third largest element in A, and A[ 2] is the fifth largest element in A. 
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Design an algorithm for computing the kth largest element in an array. Assume 
entries are distinct. 

Hint: Use divide and conquer in conjunction with randomization. 

Solution: The brute-force approach is to sort the input array A in descending order 
and return the element at index k - 1. The time complexity is 0(n 1 ogn), where n is 
the length of A. 

Sorting is wasteful, since it does more than what is required. For example, if we 
want the first largest element, we can compute that with a single iteration, which is 
0(n). 

For general k, we can store a candidate set of k elements in a min-heap, in a fashion 
analogous to Solution 11.4 on Page 181, which will yield a 0(n log k) time complexity 
and 0{k) space complexity. This approach is faster than sorting but is not in-place. 
Additionally, it does more than what's required—it computes the k largest elements 
in sorted order, but all that's asked for is the kth largest element. 

Conceptually, to focus on the kth largest element in-place without completely 
sorting the array we can select an element at random (the "pivot"), and partition 
the remaining entries into those greater than the pivot and those less than the pivot. 
(Since the problem states all elements are distinct, there cannot be any other elements 
equal to the pivot.) If there are exactly k - 1 elements greater than the pivot, the pivot 
must be the kth largest element. If there are more than k - 1 elements greater than the 
pivot, we can discard elements less than or equal to the pivot—the fc-largest element 
must be greater than the pivot. If there are less than k - 1 elements greater than the 
pivot, we can discard elements greater than or equal to the pivot. 

Intuitively, this is a good approach because on average we reduce by half the 
number of entries to be considered. 

Implemented naively, this approach requires 0(n) additional memory. However, 
we can avoid the additional storage by using the array itself to record the partitioning. 

private static class Compare { 

private static class GreaterThan implements Comparator<Integer> { 
public int compare(Integer a, Integer b) { 
return (a > b) ? -1 : (a.equals(b)) ? © : 1; 

} 

} 

public static final GreaterThan GREATER_THAN = new GreaterThan(); 

} 

// The numbering starts from one, i.e., if A = [3, 1,-1,2] then 
// findKthLargest(A, 1) returns 3, findKthLargest(A, 2) returns 2, 

// findKthLargest(A, 3) returns 1, and findKthLargest(A, 4) returns -1. 
public static int findKthLargest(Listdnteger> A, int k) { 
return findKth(A, k, Compare.GREATER_THAN); 

} 

public static int findKth(List<Integer> A, int k, Comparator<Integer> cmp) { 
int left = 8, right = A.sizeO - 1; 

Random r = new Random(8); 
while (left <= right) { 
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// Generates a random integer in [left, right]. 
int pivotldx = r.nextlnt(right - left + 1) + left; 

int newPivotldx = partitionAroundPivot(left, right, pivotldx, A, cmp); 
if (newPivotldx == k - 1) { 
return A.get(newPivotldx); 

} else if (newPivotldx > k - 1) { 
right = newPivotldx - 1; 

} else { // newPivotldx < k - 1. 
left = newPivotldx + 1; 

} 

} 

} 

// Partitions A.subList(left, right+1) around pivotldx, returns the new index 
// of the pivot, newPivotldx, after partition. After partitioning, 

// A.subList(left, newPivotldx) contains elements that are less than the 
// pivot, and A.subList(newPivotldx + 1 , right + 1) contains elements that 
// are greater than the pivot. 

// 

// Note: "less than" is defined by the Comparator object. 

// 

// Returns the new index of the pivot element after partition. 

private static int partitionAroundPivot(int left, int right, int pivotldx, 

List<Integer> A, 

Comparator<Integer> cmp) { 

int pivotValue = A.get(pivotldx); 
int newPivotldx = left; 

Collections.swap(A, pivotldx, right); 
for (int i = left; i < right; ++i) { 

if (cmp.compare(A.get(i), pivotValue) < ®) { 

Collections.swap(A, i, newPivotldx++); 

> 

> 

Collections.swap(A, right, newPivotldx); 
return newPivotldx; 

} 

Since we expect to reduce the number of elements to process by roughly half, the 
average time complexity T(n) satisfies T(n) = 0(n)+T(n/ 2). This solves to T(n) = 0(n). 
The space complexity is 0(1). The worst-case time complexity is 0(n 2 ), which occurs 
when the randomly selected pivot is the smallest or largest element in the current 
subarray. The probability of the worst-case reduces exponentially with the length 
of the input array, and the worst-case is a nonissue in practice. For this reason, the 
randomize selection algorithm is sometimes said to have almost certain 0(n) time 
complexity. 

Variant: Design an algorithm for finding the median of an array. 

Variant: Design an algorithm for finding the fcth largest element of A in the presence 
of duplicates. The A:th largest element is defined to be A [k - 1] after A has been sorted 
in a stable manner, i.e., if A[i] = A[j] and i < j then A[i] must appear before A[j] after 
stable sorting. 

Variant: A number of apartment buildings are coming up on a new street. The postal 
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service wants to place a single mailbox on the street. Their objective is to minimize 
the total distance that residents have to walk to collect their mail each day. (Different 
buildings may have different numbers of residents.) 

Devise an algorithm that computes where to place the mailbox so as to minimize the 
total distance, that residents travel to get to the mailbox. Assume the input is specified 
as an array of building objects, where each building object has a field indicating the 
number of residents in that building, and a field indicating the building's distance 
from the start of the street. 


12.9 Find the missing IP address 

The storage capacity of hard drives dwarfs that of RAM. This can lead to interesting 
space-time trade-offs. 

Suppose you were given a file containing roughly one billion IP addresses, each of 
which is a 32-bit quantity. How would you programmatically find an IP address that 
is not in the file? Assume you have unlimited drive space but only a few megabytes 
of RAM at your disposal. 

Hint: Can you be sure there is an address which is not in the file? 

Solution: Since the file can be treated as consisting of 32-bit integers, we can sort the 
input file and then iterate through it, searching for a gap between values. The time 
complexity is 0(n log n), where n is number of entries. Furthermore, to keep the RAM 
usage low, the sort will have to use disk as storage, which in practice is very slow. 

Note that we cannot just compute the largest entry and add one to it, since if the 
largest entry is 255.255.255.255 (the highest possible IP address), adding one to it 
leads to overflow. The same holds for the smallest entry. (In practice this would be a 
good heuristic.) 

We could add all the IP addresses in the file to a hash table, and then enumerate IP 
addresses, starting with 0.0.0.0, until we find one not in the hash table. However, a 
hash table requires considerable overhead—of the order of 10 bytes for each integer, 
i.e., of the order of 10 gigabytes. 

We can reduce the storage requirement by an order of magnitude by using a bit 
array representation for the set of all possible IP addresses. Specifically, we allocate 
an array of 2 32 bits, initialized to 0, and write a 1 at each index that corresponds to an 
IP address in the file. Then we iterate through the bit array, looking for an entry set 
to 0. There are 2 32 « 4 X 10 9 possible IP addresses, so not all IP addresses appear in 
the file. The storage is 2 32 /8 bytes, is half a gigabyte. This is still well in excess of the 
storage limit. 

Since the input is in a file, we can make multiple passes through it. We can use 
this to narrow the search down to subsets of the space of all IP addresses as follows. 
We make a pass through the file to count the number of IP addresses present whose 
leading bit is a 1, and the number of IP addresses whose leading bit is a 0. At least 
one IP address must exist which is not present in the file, so at least one of these two 
counts is below 2 31 . For example, suppose we have determined using counting that 
there must be an IP address which begins with 0 and is absent from the file. We can 
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focus our attention on IP addresses in the file that begin with 0, and continue the 
process of elimination based on the second bit. This entails 32 passes, and uses only 
two integer-valued count variables as storage. 

Since we have more storage, we can count on groups of bits. Specifically, we can 
count the number of IP addresses in the file that begin with 0,1,2,..., 2 16 - 1 using 
an array of 2 16 32-bit integers. For every IP address in the file, we take its 16 MSBs to 
index into this array and increment the count of that number. Since the file contains 
fewer than 2 32 numbers, there must be one entry in the array that is less than 2 16 . This 
tells us that there is at least one IP address which has those upper bits and is not in 
the file. In the second pass, we can focus only on the addresses whose leading 16 
bits match the one we have found, and use a bit array of size 2 16 to identify a missing 
address. 

private static final int NUM_BUCKET = 1 « 16; 

public static int findMissingElement(Iterable<Integer> sequence) { 
int[] counter = new int[NUM_BUCKET]; 

Iterator<Integer> s = sequence.iterator(); 
while (s.hasNext()) { 

int idx = s.nextO »> 16; 

++counter[idx]; 

} 

for (int i = Q; i < counter.length; ++i) { 

// Look for a bucket that contains less than NUM_BUCKET elements. 
if (counter[i] < NUM_BUCKET) { 

BitSet bitVec = new BitSet(NUM.BUCKET); 

s = sequence.iterator() ; // Search from the beginning again. 
while (s.hasNext()) { 
int x = s . next () ; 
if (i == (x »> 16)) { 

bitVec.set(((NUM_BUCKET)-1) & x); // Gets the lower 16 bits of x. 

} 

} 

for (int j = ®; j < (1 « 16); ++j) { 
if (!bitVec.get(j)) { 
return (i « 16) | j; 

} 

} 

} 

} 

} 


The storage requirement is dominated by the count array, i.e., 2 16 4 byte entries, which 
is a quarter of a megabyte. 


12.10 Find the duplicate and missing elements 

If an array contains n — 1 integers, each between 0 and n — 1, inclusive, and all numbers 
in the array are distinct, then it must be the case that exactly one number between 0 
and n — 1 is absent. 
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We can determine the missing number in 0(n) time and 0(1) space by computing 
the sum of the elements in the array. Since the sum of all the numbers from 0 to n - 1, 
inclusive, is , we can subtract the sum of the numbers in the array from to 
get the missing number. 

For example, if the array is (5,3,0,1,2), then n = 6. We subtract (5+3+0+1+ 2) = 11 
from = 15, and the result, 4, is the missing number. 

Similarly, if the array contains n + 1 integers, each between 0 and n — 1, inclusive, 
with exactly one element appearing twice, the duplicated integer will be equal to the 
sum of the elements of the array minus . 

Alternatively, for the first problem, we can compute the missing number by com¬ 
puting the XOR of all the integers from 0 to n - 1, inclusive, and XORing that with the 
XOR of all the elements in the array. Every element in the array, except for the missing 
element, cancels out with an integer from the first set. Therefore, the resulting XOR 
equals the missing element. The same approach works for the problem of finding 
the duplicated element. For example, the array (5,3,0,1,2) represented in binary is 
<(101) 2 , (011) 2 , (000) 2 , (001) 2 , (010) 2 >. The XOR of these entries is (101) 2 . The XOR of all 
numbers from 0 to 5, inclusive, is (001) 2 . The XOR of (101) 2 and (001) 2 is (100) 2 = 4, 
which is the missing number. 

We now turn to a related, though harder, problem. 

You are given an array of n integers, each between 0 and n - 1, inclusive. Exactly 
one element appears twice, implying that exactly one number between 0 and n - 1 
is missing from the array. How would you compute the duplicate and missing 
numbers? 

Hint: Consider performing multiple passes through the array. 

Solution: A brute-force approach is to use a hash table to store the entries in the 
array. The number added twice is the duplicate. After having built the hash table, 
we can test for the missing element by iterating through the numbers from 0 to 
n — 1, inclusive, stopping when a number is not present in the hash table. The time 
complexity and space complexity are 0(n). We can improve the space complexity to 
0(1) by sorting the array, subsequent to which finding duplicate and missing values 
is trivial. However, the time complexity increases to 0(n log n). 

We can improve on the space complexity by focusing on a collective property of 
the numbers in the array, rather than the individual numbers. For example, let t be 
the element appearing twice, and m be the missing number. The sum of the numbers 
from 0 to n - 1, inclusive, is , so the sum of the elements in the array is exactly 
(w ~ 1)w +t-m. This gives us an equation in t and m, but we need one more independent 
equation to solve for them. 

We could use an equation for the product of the elements in the array, or for the 
sum of the squares of the elements in the array. Both of these are unsatisfactory 
because they are prone to overflow. 

The introduction to this problem showed how to find a missing number from an 
array of n - 2 distinct numbers between 0 and n - 1 using XOR. Applying the same 
idea to the current problem, i.e., computing the XOR of all the numbers from 0 to 
n — 1, inclusive, and the entries in the array, yields m®t. This does not seem very 
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helpful at first glance, since we want m and t. However, since m ± t, there must be 
some bit in m © t that is set to 1, i.e., m and t differ in that bit. For example, the XOR of 
(01101)2 and (11100)2 is (10001)2- The Is in the XOR are exactly the bits where (01101)2 
and (11100)2 differ. 

This fact allows us to focus on a subset of numbers from 0 to n - 1 where we can 
guarantee exactly one of m and t is present. Suppose we know m and t differ in the 
kth bit. We compute the XOR of the numbers from 0 to n - 1 in which the kth bit is 1, 
and the entries in the array in which the kth. bit is 1. Let this XOR be h —by the logic 
described in the problem statement, h must be one of m or t. We can make another 
pass through A to determine if h is the duplicate or the missing element. 

For example, for the array (5,3,0,3,1,2), the duplicate entry f is 3 
and the missing entry m is 4. Represented in binary the array is 
((101)2,(011)2,(000)2,(011)2,(001)2,(010)2). The XOR of these entries is (110) 2 . The 
XOR of the numbers from 0 to 5, inclusive, is (001)2. The XOR of (110)2 and (001)2 
is (111) 2 . This tells we can focus our attention on entries where the least significant 
bit is 1. We compute the XOR of all numbers between 0 and 5 in which this bit is 
1, i.e., (001)2, (011)2/ and (101)2, and all entries in the array in which this bit is 1, i.e., 
(101) 2 , (011) 2 , (011)2/ and (001) 2 . The XOR of these seven values is (011) 2 . This implies 
that (011)2 = 3 is either the missing or the duplicate entry. Another pass through 
the array shows that it is the duplicate entry. We can then find the missing entry by 
forming the XOR of (011) 2 with all entries in the array, and XORing that result with 
the XOR of all numbers from 0 to 5, which yields (100)2, i.e., 4. 

private static class DuplicateAndMissing { 
public Integer duplicate; 
public Integer missing; 

public DuplicateAndMissing(Integer duplicate, Integer missing) { 
this.duplicate = duplicate; 
this.missing = missing; 

} 

} 

public static DuplicateAndMissing findDuplicateMissing(List<Integer> A) { 

// Compute the XOR of all numbers from 9 to /A/ - 1 and all entries in A. 
int missXORDup = ©; 

for (int i = Q; i < A.sizeO; ++i) { 
missXORDup A = i A A.get(i); 

} 

// We need to find a bit that’s set to 1 in missXORDup. Such a bit 
// must exist if there is a single missing number and a single duplicated 
// number in A . 

// 

// The bit-fiddling assignment below sets all of bits in differBit to 9 
// except for the least significant bit in missXORDup that’s 1. 
int differBit = missXORDup & (~(missX0RDup - 1)); 
int missOrDup = ©; 

for (int i = ©; i < A.sizeO; ++i) { 

// Focus on entries and numbers in which the differBit-th bit is 1. 
if ((i & differBit) != ®) { 
missOrDup A = i; 
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} 

if ((A.get(i) & differBit) != ®) { 
missOrDup A = A.get(i); 

} 

} 

// missOrDup is either the missing value or the duplicated entry. 
for (int a : A) { 

if (a == missOrDup) { // missOrDup is the duplicate. 

return new DuplicateAndMissing(missOrDup, missOrDup A missXORDup); 

} 

} 

// missOrDup is the missing value. 

return new DuplicateAndMissing(missOrDup A missXORDup, missOrDup); 

} 

The time complexity is 0(n) and the space complexity is 0(1). 
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Chapter 


Hash Tables 


The new methods are intended to reduce the amount of space re¬ 
quired to contain the hash-coded information from that associated 
with conventional methods. The reduction in space is accomplished 
by exploiting the possibility that a small fraction of errors of com¬ 
mission may be tolerable in some applications. 

— "Space/time trade-offs in hash coding with allowable errors," 

B. H. Bloom, 1970 


The idea underlying a hash table is to store objects according to their key field in an 
array. Objects are stored in array locations ("slots") based on the "hash code" of the 
key. The hash code is an integer computed from the key by a hash function. If the 
hash function is chosen well, the objects are distributed uniformly across the array 
locations. 

If two keys map to the same location, a "collision" is said to occur. The standard 
mechanism to deal with collisions is to maintain a linked list of objects at each array 
location. If the hash function does a good job of spreading objects across the un¬ 
derlying array and take 0( 1) time to compute, on average, lookups, insertions, and 
deletions have 0(1 + n/m ) time complexity, where n is the number of objects and m 
is the length of the array. If the "load" rc/m grows large, rehashing can be applied 
to the hash table. A new array with a larger number of locations is allocated, and 
the objects are moved to the new array. Rehashing is expensive (0(n + m) time) but 
if it is done infrequently (for example, whenever the number of entries doubles), its 
amortized cost is low. 

A hash table is qualitatively different from a sorted array—keys do not have to ap¬ 
pear in order, and randomization (specifically, the hash function) plays a central role. 
Compared to binary search trees (discussed in Chapter 15), inserting and deleting in a 
hash table is more efficient (assuming rehashing is infrequent). One disadvantage of 
hash tables is the need for a good hash function but this is rarely an issue in practice. 
Similarly, rehashing is not a problem outside of realtime systems and even for such 
systems, a separate thread can do the rehashing. 

A hash function has one hard requirement—equal keys should have equal hash 
codes. This may seem obvious, but is easy to get wrong, e.g., by writing a hash 
function that is based on address rather than contents, or by including profiling data. 

A softer requirement is that the hash function should "spread" keys, i.e., the hash 
codes for a subset of objects should be uniformly distributed across the underlying 
array. In addition, a hash function should be efficient to compute. 


207 




A common mistake with hash tables is that a key that's present in a hash table will 
be updated. The consequence is that a lookup for that key will now fail, even though 
it's still in the hash table. If you have to update a key, first remove it, then update it, 
and finally, add it back—this ensures it's moved the correct array location. As a rule, 
you should avoid using mutable objects as keys. 

Now we illustrate the steps for designing a hash function suitable for strings. First, 
the hash function should examine all the characters in the string. It should give a 
large range of values, and should not let one character dominate (e.g., if we simply 
cast characters to integers and multiplied them, a single 0 would result in a hash code 
of 0). We would also like a rolling hash function, one in which if a character is deleted 
from the front of the string, and another added to the end, the new hash code can be 
computed in (9(1) time (see Solution 7.13 on Page 109). The following function has 
these properties: 

public static int stringHash(String str, int modulus) { 
int kMult = 997; 
int val = ®; 

for (int i = ®; i < str.length(); i++) { 
char c = str.charAt(i); 
val = (val * kMult + c) % modulus; 

} 

return val; 

} 


A hash table is a good data structure to represent a dictionary, i.e., a set of strings. 
In some applications, a trie, which is a tree data structure that is used to store a 
dynamic set of strings, has computational advantages. Unlike a BST, nodes in the tree 
do not store a key. Instead, the node's position in the tree defines the key which it is 
associated with. See Problem 25.20 on Page 467 for an example of trie construction 
and application. 

Hash tables boot camp 

We introduce hash tables with two examples—one is an application that benefits from 
the algorithmic advantages of hash tables, and the other illustrates the design of a 
class that can be used in a hash table. 

An application of hash tables 

Anagrams are popular word play puzzles, whereby rearranging letters of one set of 
words, you get another set of words. For example, "eleven plus two" is an anagram 
for "twelve plus one". Crossword puzzle enthusiasts and Scrabble players benefit 
from the ability to view all possible anagrams of a given set of letters. 

Suppose you were asked to write a program that takes as input a set of words and 
returns groups of anagrams for those words. Each group must contain at least two 
words. 

For example, if the input is "debitcard", "elvis", "silent", "badcredit", "lives", 
"freedom", "listen", "levis", "money" then there are three groups of ana¬ 
grams: (1.) "debitcard", "badcredit"; (2.) "elvis", "lives", "levis"; (3.) "silent", "listen". 
(Note that "money" does not appear in any group, since it has no anagrams in the 
set.) 
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Let's begin by considering the problem of testing whether one word is an anagram 
of another. Since anagrams do not depend on the ordering of characters in the 
strings, we can perform the test by sorting the characters in the string. Two words 
are anagrams if and only if they result in equal strings after sorting. For example, 
sort('Togarithmic") and sort( "algorithmic") are both "acghiilmort", so "logarithmic" 
and "algorithmic" are anagrams. 

We can form the described grouping of strings by iterating through all strings, and 
comparing each string with all other remaining strings. If two strings are anagrams, 
we do not consider the second string again. This leads to an 0(n 2 m logra) algorithm, 
where n is the number of strings and m is the maximum string length. 

Looking more carefully at the above computation, note that the key idea is to 
map strings to a representative. Given any string, its sorted version can be used as a 
unique identifier for the anagram group it belongs to. What we want is a map from 
a sorted string to the anagrams it corresponds to. Anytime you need to store a set of 
strings, a hash table is an excellent choice. Our final algorithm proceeds by adding 
sort(s) for each string s in the dictionary to a hash table. The sorted strings are keys, 
and the values are arrays of the corresponding strings from the original input. 

public static List<List<String» findAnagrams (List<String> dictionary) { 

Map<String , List<String» sortedStringToAnagrams = new HashMap<>() ; 

for (String s : dictionary) { 

// Sorts the string, uses it as a key, and then appends 
// the original string as another value in the hash table. 
char[] sortedCharArray = s.toCharArray(); 

Arrays.sort(sortedCharArray); 

String sortedStr = new String(sortedCharArray); 

if (!sortedStringToAnagrams.containsKey(sortedStr)) { 

sortedStringToAnagrams.put(sortedStr, new ArrayList<String>()); 

> 

sortedStringToAnagrams.get(sortedStr).add(s); 

} 

List<List<String» anagramGroups = new ArrayList<>() ; 

for (Map. Entry<String , List<String» p : 
sortedStringToAnagrams.entrySet()) { 
if (p.getValue().size() >= 2) { // Found anagrams. 
anagramGroups.add(p.getValue()); 

} 

} 

return anagramGroups; 

} 


The computation consists of n calls to sort and n insertions into the hash table. Sorting 
all the keys has time complexity 0(nm log m). The insertions add a time complexity 
of 0(nm), yielding 0(nm log m) time complexity in total. 

Design of a hashable class 

Consider a class that represents contacts. For simplicity, assume each contact is a 
string. Suppose it is a hard requirement that the individual contacts are to be stored 
in a list and it's possible that the list contains duplicates. Two contacts should be 
equal if they contain the same set of strings, regardless of the ordering of the strings 
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within the underlying list. Multiplicity is not important, i.e., three repetitions of the 
same contact is the same as a single instance of that contact. In order to be able to 
store contacts in a hash table, we first need to explicitly define equality, which we can 
do by forming sets from the lists and comparing the sets. 

In our context, this implies that the hash function should depend on the strings 
present, but not their ordering; it should also consider only one copy if a string 
appears in duplicate form. We do this by forming a set from the list, and calling the 
hash function for the set. The library hash function for sets is order-independent, and 
a set automatically suppress duplicates, so this hash function meets our requirements. 
It should be pointed out that the hash function and equals methods below are very 
inefficient. In practice, it would be advisable to cache the underlying set and the hash 
code, remembering to void these values on updates. 

public static List<ContactList> mergeContactLists( 

ListcContactList> contacts) { 
return new ArrayList<>(new HashSet(contacts)); 

} 

public static class ContactList { 
public List<String> names; 

ContactList(List<String> names) { this.names = names; } 

©Override 

public boolean equals(Object obj) { 

if (obj == null || !(obj instanceof ContactList)) { 
return false; 

} 

return this == obj 
? true 

: new HashSet(names).equals(new HashSet(((ContactList)obj).names)); 

} 

©Override 

public int hashCode() { 

return new HashSet(names).hashCode(); 

} 

} 


The time complexity of computing the hash is 0{n), where n is the number of 
strings in the contact list. Hash codes are often cached for performance, with the 
caveat that the cache must be cleared if object fields that are referenced by the hash 
function are updated. 

Know your hash table libraries 

There are two hash table-based data structures commonly used in Java—HashSet 
and HashMap. The difference between the two is that the latter stores key-value pairs, 
whereas the former simply stores keys. Both have the property that they do not allow 
for duplicate keys, unlike, for example, LinkedList and PriorityQueue. Technically, 
HashSet implements the Set interface, and HashMap implements the Map interface. 

The most important methods for HashSet are methods defined in Set—add( 144), 
remove(“Cantor”), contains(24), iteratorQ, isEmptyQ, and sizeQ. Both 


210 



Hash tables have the best theoretical and real-world performance for lookup, 
insert and delete. Each of these operations has 0(1) time complexity. The 0(1) 
time complexity for insertion is for the average case—a single insert can take 0(n) 
if the hash table has to be resized. [Problem 13.2] 

Consider using a hash code as a signature to enhance performance, e.g., to filter 
out candidates. [Problem 13.14] 

Consider using a precomputed lookup table instead of boilerplate if-then code 
for mappings, e.g., from character to value, or character to character. [Problem 7.9] 

When defining your own type that will be put in a hash table, be sure you under¬ 
stand the relationship between logical equality and the fields the hash function 
must inspect. Specifically, anytime equality is implemented, it is imperative that 
the correct hash function is also implemented, otherwise when objects are placed 
in hash tables, logically equivalent objects may appear in different buckets, leading 
to lookups returning false, even when the searched item is present. 

Sometimes you'H need a multimap, i.e., a map that contains multiple values for 
a single key, or a bi-directional map. If the language's standard libraries do not 
provide the functionality you need, learn how to implement a multimap using 
lists as values, or find a third party library that has a multimap. 


add(144) and remove (“Cantor”) return a boolean indicating if the added/removed 
element was already present. It's important to remember that null is a valid entry. 

• The order in which keys are traversed by the iterator returned by i terator () is 
unspecified; it may even change with time. The class LinkedHashSet subclasses 
HashSet—the only difference is that iterator() returns keys in the order in 
which they were inserted into the set. This order is not affected if an element is 
re-inserted into the set, i.e., if s. add(x) is called when s. contains(x) is true. 

• HashSet implements retainAll(C), which can be used to perform set 
intersection—this can be used to reduce coding burden substantially in some 
cases. A related method is removeAll(C). 

The most important methods for HashMap are methods defined in Map—put (’ z ’ , 
26), get(“Hardy”), removeC ’z ’), and containsKey(“Hardy”). 

• There are several methods that are relevant to iteration—entrySet (), keySet (), 
and values(). (Note that these methods do not follow a parallel naming 
scheme.) The generic static inner class Map. Entry<K, V> is used to iterate over 
key-value pairs. Iteration order is not fixed (though iteration over the entry set, 
the key set, and the value set does match up). 

• To iterate in fixed order, use a LinkedHashMap. A LinkedHashMap is somewhat 
more complex than a LinkedHashSet—for example, it can be specified that 
the iteration should proceed in insertion order, or in access order (in which 
case a call toget(42) will move the corresponding element to the front of the 
ordering). A LinkedHashMap can also specify capacity constraints, and enable 
an LRU eviction policy. 

The Objects class provides a number of static utility methods that can greatly 
reduce the burden of writing equals(obj) and hashCodeQ. Example methods 
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include Objects.equals(x,y). Objects.deepEquals(A,B), and Objects.hash(42, 
“Douglas Adams”, 3.14, new DateO). 

13.1 Test for palindromic permutations 

A palindrome is a string that reads the same forwards and backwards, e.g., "level", 
"rotator", and "foobaraboof". 

Write a program to test whether the letters forming a string can be permuted to form 
a palindrome. For example, "edified" can be permuted to form "deified". 

Hint: Find a simple characterization of strings that can be permuted to form a palindrome. 

Solution: A brute-force approach is to compute all permutations of the string, and 
test each one for palindromicity. This has a very high time complexity. Examining 
the approach in more detail, one thing to note is that if a string begins with say 
'a', then we only need consider permutations that end with 'a'. This observation 
can be used to prune the permutation-based algorithm. However, a more powerful 
conclusion is that all characters must occur in pairs for a string to be permutable into 
a palindrome, with one exception, if the string is of odd length. For example, for the 
string "edified", which is of odd length (7) there are two V, two 'f's, two 'i's, and one 
'd '—this is enough to guarantee that "edified" can be permuted into a palindrome. 

More formally, if the string is of even length, a necessary and sufficient condition 
for it to be a palindrome is that each character in the string appear an even number 
of times. If the length is odd, all but one character should appear an even number of 
times. Both these cases are covered by testing that at most one character appears an 
odd number of times, which can be checked using a hash table mapping characters 
to frequencies. 


public static boolean canFormPalindrome(String s) { 

MapcCharacter, Integer> charFrequencies = new HashMap<>(); 

// Compute the frequency of each char in s. 
for (int i = Q; i < s.length(); i++) { 
char c = s.charAt(i); 

if (!charFrequencies.containsKey(c)) { 
charFrequencies.put(c, 1); 

} else { 

charFrequencies.put(c, charFrequencies.get(c) + 1); 

} 

} 

// A string can be permuted as a palindrome if and only if the number of 
// chars whose frequencies is odd is at most 1. 
int oddCount = Q; 

for (Map.EntrycCharacter, Integer> p : charFrequencies.entrySet()) { 
if ((p.getValue() % 2) != ® && ++oddCount > 1) { 

return false; 

} 

} 

return true; 
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The time complexity is 0(n), where n is the length of the string. The space complexity 
is 0(c), where c is the number of distinct characters appearing in the string. 


13.2 Is AN ANONYMOUS LETTER CONSTRUCTIBLE? 

Write a program which takes text for an anonymous letter and text for a magazine 
and determines if it is possible to write the anonymous letter using the magazine. 
The anonymous letter can be written using the magazine if for each character in the 
anonymous letter, the number of times it appears in the anonymous letter is no more 
than the number of times it appears in the magazine. 

Hint: Count the number of distinct characters appearing in the letter. 

Solution: A brute force approach is to count for each character in the character set the 
number of times it appears in the letter and in the magazine. If any character occurs 
more often in the letter than the magazine we return false, otherwise we return true. 
This approach is potentially slow because it iterates over all characters, including 
those that do not occur in the letter or magazine. It also makes multiple passes 
over both the letter and the magazine—as many passes as there are characters in the 
character set. 

A better approach is to make a single pass over the letter, storing the character 
counts for the letter in a single hash table—keys are characters, and values are the 
number of times that character appears. Next, we make a pass over the magazine. 
When processing a character c, if c appears in the hash table, we reduce its count by 1; 
we remove it from the hash when its count goes to zero. If the hash becomes empty, 
we return true. If we reach the end of the letter and the hash is nonempty, we return 
false—each of the characters remaining in the hash occurs more times in the letter 
than the magazine. 

public static boolean isLetterConstructibleFromMagazine(String letterText, 

String magazineText) { 

Map<Character, Integer> charFrequencyForLetter = new HashMap<>(); 

// Compute the frequencies for all chars in letterText. 
for (int i = ®; i < letterText.length(); i++) { 
char c = letterText.charAt(i); 
if (!charFrequencyForLetter.containsKey(c)) { 
charFrequencyForLetter.put(c, 1); 

} else { 

charFrequencyForLetter.put(c, charFrequencyForLetter.get(c) + 1); 

} 

} 

// Check if the characters in magazineText can cover characters in 
// letterText. 

for (char c : magazineText.toCharArray()) { 
if (charFrequencyForLetter.containsKey(c)) { 

charFrequencyForLetter.put(c, charFrequencyForLetter.get(c) - 1); 
if (charFrequencyForLetter.get(c) == ®) { 
charFrequencyForLetter.remove(c); 

// All characters for letterText are matched. 
if (charFrequencyForLetter.isEmpty()) { 
break; 
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} 

} 

} 

} 

// Empty charFrequencyForLetter means every char in letterText can be 
// covered by a character in magazineText. 
return charFrequencyForLetter.isEmptyQ; 


In the worst-case, the letter is not constructible or the last character of the magazine is 
essentially required. Therefore, the time complexity is 0(m + n) where m and n are the 
number of characters in the letter and magazine, respectively. The space complexity 
is the size of the hash table constructed in the pass over the letter, i.e., 0(L), where L 
is the number of distinct characters appearing in the letter. 

If the characters are coded in ASCII, we could do away with the hash table and use 
a 256 entry integer array A, with A[i] being set to the number of times the character i 
appears in the letter. 


13.3 Implement an ISBN cache 

The International Standard Book Number (ISBN) is a unique commercial book iden¬ 
tifier. It is a string of length 10. The first 9 characters are digits; the last character 
is a check character. The check character is the sum of the first 9 digits, modulo 11, 
with 10 represented by 'X'. (Modem ISBNs use 13 digits, and the check digit is taken 
modulo 10; this problem is concerned with 10-digit ISBNs.) 

Create a cache for looking up prices of books identified by their ISBN. You implement 
lookup, insert, and remove methods. Use the Least Recently Used (LRU) policy for 
cache eviction. If an ISBN is already present, insert should not change the price, but 
it should update that entry to be the most recently used entry. Lookup should also 
update that entry to be the most recently used entry. 

Hint: Amortize the cost of deletion. Alternatively, use an auxiliary data structure. 

Solution: Hash tables are ideally suited for fast lookups. We can use a hash table to 
quickly lookup price by using ISBNs as keys. Along with each key, we store a value, 
which is the price and the most recent time a lookup was done on that key. 

This yields 0(1) lookup times on cache hits. Inserts into the cache are also 0(1) 
time, until the cache is full. Once the cache fills up, to add a new entry we have to 
find the LRU entry, which will be evicted to make place for the new entry. Finding 
this entry takes 0(n) time, where n is the cache size. 

One way to improve performance is to use lazy garbage collection. Specifically, 
let's say we want the cache to be of size n. We do not delete any entries from the hash 
table until it grows to 2 n entries. At this point we iterate through the entire hash table, 
and find the median age of items. Subsequently we discard everything below the 
median. The worst-case time to delete becomes 0(n) but it will happen at most once 
every n operations. Therefore, the amortized time to delete is 0(1). The drawback of 
this approach is the 0(n) time needed for some lookups that miss on a full cache, and 
the 0(n) increase in memory. 
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An alternative is to maintain a separate queue of keys. In the hash table we store 
for each key a reference to its location in the queue. Each time an ISBN is looked up 
and is found in the hash table, it is moved to the front of the queue. (This requires 
us to use a linked list implementation of the queue, so that items in the middle of the 
queue can be moved to the head.) When the length of the queue exceeds n, when a 
new element is added to the cache, the item at the tail of the queue is deleted from 
the cache, i.e., from the queue and the hash table. 

The Java language provides the class LinkedHashMap, which is a subclass of 
HashMap that preserves the insertion order—an iteration through a LinkedHashMap 
visits keys in the order they were inserted. By calling the appropriate constructor, we 
can ensure that any time an entry is read, it automatically moves to the front. We can 
take advantage of this class to avoid having to implement the linked list. 

public class LRUCache { 

LinkedHashMapcInteger, Integer> isbnToPrice; 

LRUCache (final int capacity) { 
this .isbnToPrice 

= new LinkedHashMapcInteger, Integer>(capacity, l.Of, true) { 

©Override 

protected boolean removeEldestEntry(Map.Entryclnteger, Integer> e) 

{ 

return this. size() > capacity; 

} 

}; 

} 

public Integer lookup(Integer key) { 
if (!isbnToPrice.containsKey(key)) { 
return null; 

> 

return isbnToPrice.get(key); 

} 

public Integer insert(Integer key, Integer value) { 

// We add the value for key only if key is not present - we don't update 
// existing values. 

Integer currentValue = isbnToPrice.get(key); 
if (!isbnToPrice.containsKey(key)) { 
isbnToPrice.put(key, value); 
return currentValue; 

} else { 

return null; 

} 

} 

public Integer erase(Object key) { return isbnToPrice.remove(key); } 


The time complexity for each lookup is 0(1) for the hash table lookup and 0(1) for 
updating the queue, i.e., 0(1) overall. 
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13.4 Compute the LCA, optimizing for close ancestors 

Problem 10.4 on Page 157 is concerned with computing the LCA in a binary tree with 
parent pointers in time proportional to the height of the tree. The algorithm presented 
in Solution 10.4 on Page 157 entails traversing all the way to the root even if the nodes 
whose LCA is being computed are very close to their LCA. 

Design an algorithm for computing the LCA of two nodes in a binary tree. The 
algorithm's time complexity should depend only on the distance from the nodes to 
the LCA. 

Hint: Focus on the extreme case described in the problem introduction. 

Solution: The brute-force approach is to traverse upwards from the one node to the 
root, recording the nodes on the search path, and then traversing upwards from the 
other node, stopping as soon as we see a node on the path from the first node. The 
problem with this approach is that if the two nodes are far from the root, we end up 
traversing all the way to the root, even if the LCA is the parent of the two nodes, i.e., 
they are siblings. This is illustrated in by L and N in Figure 10.1 on Page 150. 

Intuitively, the brute-force approach is suboptimal because it potentially processes 
nodes well above the LCA. We can avoid this by alternating moving upwards from 
the two nodes and storing the nodes visited as we move up in a hash table. Each time 
we visit a node we check to see if it has been visited before. 

public static BinaryTree<Integer> LCA(BinaryTree<Integer> node®, 

BinaryTree<Integer> nodel) { 
Set<BinaryTree<Integer>> hash = new HashSet<>(); 
while (node® != null || nodel != null) { 

// Ascend tree in tandem from these two nodes. 
if (node® != null) { 

if (!hash.add(node®)) { 
return node®; 

} 

node® = node®.parent; 

} 

if (nodel != null) { 

if (!hash.add(nodel)) { 
return nodel; 

} 

nodel = nodel.parent; 

} 

} 

throw new IllegalArgumentException( 

"node® and nodel are not in the same tree"); 

} 


Note that we are trading space for time. The algorithm for Solution 10.4 on Page 157 
used 0(1) space and 0(h) time, whereas the algorithm presented above uses O(D0+Dl) 
space and time, where DO is the distance from the LCA to the first node, and D1 is 
the distance from the LCA to the second node. In the worst-case, the nodes are leaves 
whose LCA is the root, and we end up using 0(h) space and time, where h is the 
height of the tree. 
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13.5 Compute the k most frequent queries 

You are given a log file containing search queries. Each query is a string, and queries 
are separated by newlines. Diverse applications, such as autocompletion and trend 
analysis, require computing the most frequent queries. In the abstract, you are to 
solve the following problem. 

You are given an array of strings. Compute the k strings that appear most frequently 
in the array. 

Hint: Consider extreme values for k, as well as scenarios where there are a relatively small 
number of distinct strings. 

Solution: The brute-force approach is to first find the distinct strings and how often 
each one of them occurs using a hash table—keys are strings and values are frequen¬ 
cies. After building the hash table, we create a new array on the unique strings and 
sort the new array, using a custom comparator in which strings are ordered by their 
frequency (which we get via a lookup in the hash table). The k top entries in the new 
array after sorting are the result. The time complexity is 0(n + m log m), where n is the 
number of strings the original array, and m is the number of distinct strings. The first 
term comes from building the hash table, and the second comes from the complexity 
to sort the array. The space complexity is 0{m). 

Since all that is required is the k most frequent strings, the sort phase in above 
algorithm is overkill because it tells us about the relative frequencies of strings that 
are infrequent too. We can achieve a time complexity of 0(n + mlogk). This approach 
is superior when m is large, i.e., comparable to n, and k is small. We do this by 
maintaining a min-heap of the k most frequent strings. We add the first k strings 
to the hash table. We compare the frequency of each subsequent string with the 
frequency of the string at the root of the min-heap. If the new string's frequency is 
greater than the root's frequency, we delete the root and add the new string to the 
min-heap. The k strings in the min-heap at the end of the iteration are the result. In the 
worst-case, each iterative step entails a heap delete and insert, so the time complexity 
is 0(n + m log/c). The space complexity is dominated by the hash table, i.e., 0(m). 

We can improve the time complexity to almost certain 0(n + m) = 0(n ) by using 
the algorithm in Solution 12.8 on Page 200 to compute the k largest elements in the 
array of unique strings, again comparing strings on their frequencies. The space 
complexity is 0(m). 

13.6 Find the nearest repeated entries in an array 

People do not like reading text in which a word is used multiple times in a short 
paragraph. You are to write a program which helps identify such a problem. 

Write a program which takes as input an array and finds the distance between a 
closest pair of equal entries. For example, if s = ("All", "work", "and", "no", "play", 
"makes", "for", "no", "work", "no", "fun", "and", "no", "results"), then the second 
and third occurrences of "no" is the closest pair. 

Hint: Each entry in the array is a candidate. 
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Solution: The brute-force approach is to iterate over all pairs of entries, check if they 
are the same, and if so, if the distance between them is less than the smallest such 
distance seen so far. The time complexity is 0(n 2 ), where n is the array length. 

We can improve upon the brute-force algorithm by noting that when examining an 
entry, we do not need to look at every other entry—we only care about entries which 
are the same. We can store the set of indices corresponding to a given value using a 
hash table and iterate over all such sets. However, there is a better approach—when 
processing an entry, all we care about is the closest previous equal entry. Specifically, 
as we scan through the array, we record for each value seen so far, we store in a hash 
table the latest index at which it appears. When processing the element, we use the 
hash table to see the latest index less than the current index holding the same value. 

For the given example, when processing the element at index 9, which is "no", the 
hash table tells us the most recent previous occurrence of "no" is at index 7, so we 
update the distance of the closest pair of equal entries seen so far to 2. 

public static int findNearestRepetition(List<String> paragraph) { 

Map<String, Integer> wordToLatestlndex = new HashMap<>(); 
int nearestRepeatedDistance = Integer.MAX_VALUE; 
for (int i = ®; i < paragraph.size C); ++i) { 

if (wordToLatestlndex.containsKey(paragraph.get(i))) { 
nearestRepeatedDistance 

= Math.min(nearestRepeatedDistance, 

i - wordToLatestlndex.get(paragraph.get(i))); 

} 

wordToLatestlndex.put(paragraph.get(i), i); 

} 

return nearestRepeatedDistance; 

} 


The time complexity is 0(n), since we perform a constant amount of work per entry. 
The space complexity is 0{d), where d is the number of distinct entries in the array. 


13.7 Find the smallest subarray covering all values 

When you type keywords in a search engine, the search engine will return results, and 
each result contains a digest of the web page, i.e., a highlighting within that page of 
the keywords that you searched for. For example, a search for the keywords "Union" 
and "save" on a page with the text of the Emancipation Proclamation should return 
the result shown in Figure 13.1. 


My paramount object in this struggle is to save the Union , and is not either to 
save or to destroy slavery. If I could save the Union without freeing any slave I 
would do it, and if I could save it by freeing all the slaves I would do it; and if I 
could save it by freeing some and leaving others alone I would also do that. 

Figure 13.1: Search result with digest in boldface and search keywords underlined. 


The digest for this page is the text in boldface, with the keywords underlined for 
emphasis. It is the shortest substring of the page which contains all the keywords in 
the search. The problem of computing the digest is abstracted as follows. 
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Write a program which takes an array of strings and a set of strings, and return the 
indices of the starting and ending index of a shortest subarray of the given array that 
"covers" the set, i.e., contains all strings in the set. 

Hint: What is the maximum number of minimal subarrays that can cover the query? 

Solution: The brute force approach is to iterate over all subarrays, testing if the 
subarray contains all strings in the set. If the array length is n, there are 0(n 2 ) 
subarrays. Testing whether the subarray contains each string in the set is an 0(n) 
operation using a hash table to record which strings are present in the subarray. The 
overall time complexity is 0(n 3 ). 

We can improve the time complexity to 0(n 2 ) by growing the subarrays incremen¬ 
tally. Specifically, we can consider all subarrays starting at i in order of increasing 
length, stopping as soon as the set is covered. We use a hash table to record which 
strings in the set remain to be covered. Each time we increment the subarray length, 
we need (9(1) time to update the set of remaining strings. 

We can further improve the algorithm by noting that when we move from i to i +1 
we can reuse the work performed from i. Specifically, let's say the smallest subarray 
starting at i covering the set ends at ;. There is no point in considering subarrays 
starting at i + 1 and ending before j, since we know they cannot cover the set. When 
we advance to i + 1 , either we still cover the set, or we have to advance j to cover the 
set. We continuously advance one of i or j, which implies an 0(n) time complexity. 

As a concrete example, consider the array (apple, banana, apple, apple, dog, cat, apple, 
dog, banana, apple, cat, dog) and the set {banana, cat). The smallest subarray covering 
the set starting at 0 ends at 5. Next, we advance to 1. Since the element at 0 is not 
in the set, the smallest subarray covering the set still ends at 5. Next, we advance to 
2. Now we do not cover the set, so we advance from 5 to 8—now the subarray from 
2 to 8 covers the set. We update the start index from 2 to 3 to 4 to 5 and continue to 
cover the set. When we advance to 6, we no longer cover the set, so we advance the 
end index till we get to 10. We can advance the start index to 8 and still cover the set. 
After we move past 8, we cannot cover the set. The shortest subarray covering the 
set is from 8 to 10. 


// Represent subarray by starting and ending indices, inclusive. 
private static class Subarray { 
public Integer start; 
public Integer end; 

public Subarray(Integer start, Integer end) { 
this .start = start; 
this.end = end; 

} 

} 

public static Subarray findSmallestSubarrayCoveringSet(List<String> paragraph, 

Set<String> keywords) { 

Map<String, Integer> keywordsToCover = new HashMap<>(); 
for (String keyword : keywords) { 

keywordsToCover.put(keyword, keywordsToCover.containsKey(keyword) 

? keywordsToCover.get(keyword) + 1 

: i); 
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} 


Subarray result = new Subarray(-1, -1); 
int remainingToCover = keywords.size () ; 

for (int left = 8, right = 8; right < paragraph.size(); ++right) { 

Integer keywordCount = keywordsToCover.get(paragraph.get(right)); 
if (keywordCount != null) { 

keywordsToCover.put(paragraph.get(right), --keywordCount); 
if (keywordCount >= ®) { 

--remainingToCover; 

} 

} 

// Keeps advancing left until it reaches end or keywordsToCover does not 

// have all keywords. 

while (remainingToCover == 8) { 

if ((result.start == -1 && result.end == -1) 

|| right - left < result.end - result.start) { 
result.start = left; 
result.end = right; 

} 

keywordCount = keywordsToCover.get(paragraph.get(left)); 
if (keywordCount != null) { 

keywordsToCover.put(paragraph.get(left), ++keywordCount); 
if (keywordCount > 8) { 

++remainingToCover; 

> 

} 

++left; 

} 

> 

return result; 

} 


The complexity is 0{n), where n is the length of the array, since for each of the two 
indices we spend 0(1) time per advance, and each is advanced at most n — 1 times. 

The disadvantage of this approach is that we need to keep the subarrays in memory 
We can achieve a streaming algorithm by keeping track of latest occurrences of query 
keywords as we process A. We use a doubly linked list L to store the last occurrence 
(index) of each keyword in O, and hash table H to map each keyword in O to the 
corresponding node in L. Each time a word in O is encountered, we remove its node 
from L (which we find by using H), create a new node which records the current index 
in A, and append the new node to the end of L. We also update H. By doing this, 
each keyword in L is ordered by its order in A; therefore, if L has Hq words (i.e., all 
keywords are shown) and the current index minus the index stored in the first node 
in L is less than current best, we update current best. The complexity is still 0(n). 


// Represent subarray by starting and ending indices, inclusive . 
private static class Subarray { 
public Integer start; 
public Integer end; 

public Subarray(Integer start, Integer end) { 
this . start = start ; 
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this.end = end; 


} 

} 

private static Integer getValueForFirstEntry( 

LinkedHashMap<String, Integer> m) { 

// LinkedHashMap guarantees iteration over key-value pairs takes place in 
// insertion order, most recent first. 

Integer result = null; 

for (Map.Entry<String, Integer> entry : m.entrySet()) { 
result = entry.getValue(); 

break; 

} 

return result; 


public static Subarray findSmallestSubarrayCoveringSubset( 

Iterator<String> iter, List<String> queryStrings) { 
LinkedHashMap<String, Integer> diet = new LinkedHashMap<>(); 
for (String s : queryStrings) { 
dict.put(s, null); 

} 

int numStringsFromQueryStringsSeenSoFar = ®; 

Subarray res = new Subarray(-1, -1); 
int idx = ®; 

while (iter.hasNext()) { 

String s = iter.next(); 

if (diet.containsKey(s)) {// s is in queryStrings. 

Integer it = dict.get(s); 
if (it == null) { 

// First time seeing this string from queryStrings. 
numStringsFromQueryStringsSeenSoFar++; 

} 

// diet.put(s,idx) won't work because it does not move the entry to 
// the front of the queue if an entry with key s is already present. 
// So we explicitly remove the existing entry with key s, then put 
// (s,idx). 
diet.remove(s) ; 
diet.put(s, idx) ; 

} 

if (numStringsFromQueryStringsSeenSoFar == queryStrings.size()) { 

// We have seen all strings in queryStrings, let's get to work. 
if ((res.start == -1 && res.end == -1) 

|| idx - getValueForFirstEntry(diet) < res.end - res.start) { 
res. start = getValueForFirstEntry(diet); 
res.end = idx; 

} 

} 

++idx; 

} 

return res; 


Variant: Given an array A, find a shortest subarray A[i : j] such that each distinct 
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value present in A is also present in the subarray. 

Variant: Given an array A, rearrange the elements so that the shortest subarray 
containing all the distinct values in A has maximum possible length. 

Variant: Given an array A and a positive integer k, rearrange the elements so that no 
two equal elements are k or less apart. 


13.8 Find smallest subarray sequentially covering all values 

In Problem 13.7 on Page 218 we did not differentiate between the order in which 
keywords appeared. If the digest has to include the keywords in the order in which 
they appear in the search textbox, we may get a different digest. For example, for the 
search keywords "Union" and "save", in that order, the digest would be " Union, and 
is not either to save ". 

Write a program that takes two arrays of strings, and return the indices of the start¬ 
ing and ending index of a shortest subarray of the first array (the "paragraph" 
array) that "sequentially covers", i.e., contains all the strings in the second array 
(the "keywords" array), in the order in which they appear in the keywords array. 
You can assume all keywords are distinct. For example, let the paragraph array be 
(apple,banana, cat, apple), and the keywords array be (banana, apple). The para¬ 
graph subarray starting at index 0 and ending at index 1 does not fulfill the speci¬ 
fication, even though it contains all the keywords, since they do not appear in the 
specified order. On the other hand, the subarray starting at index 1 and ending at 
index 3 does fulfill the specification. 

Hint: For each index in the paragraph array, compute the shortest subarray ending at that index 
which fulfills the specification. 

Solution: The brute-force approach is to iterate over all subarrays of the paragraph 
array. To check whether a subarray of the paragraph array sequentially covers the 
keyword array, we search for the first occurrence of the first keyword. We never need 
to consider a later occurrence of the first keyword, since subsequent occurrences do 
not give us any additional power to cover the keywords. Next we search for the 
first occurrence of the second keyword that appears after the first occurrence of the 
first keyword. No earlier occurrence of the second keyword is relevant, since those 
occurrences can never appear in the correct order. This observation leads to an 0(n) 
time algorithm for testing whether a subarray fulfills the specification, where n is the 
length of the paragraph array. Since there are 0(n 2 ) subarrays of the paragraph array, 
the overall time complexity is 0(n 3 ). 

The brute-force algorithm repeats work. We can improve the time complexity to 
0(n 2 ) by computing for each index, the shortest subarray starting at that index which 
sequentially covers the keyword array. The idea is that we can compute the desired 
subarray by advancing from the start index and marking off the keywords in order. 

The improved algorithm still repeats work—as we advance through the paragraph 
array, we can reuse our computation of the earliest occurrences of keywords. To do 
this, we need auxiliary data structures to record previous results. 
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Specifically, we use a hash table to map keywords to their most recent occurrences 
in the paragraph array as we iterate through it, and a hash table mapping each 
keyword to the length of the shortest subarray ending at the most recent occurrence 
of that keyword. 

These two hash tables give us is the ability to determine the shortest subarray 
sequentially covering the first k keywords given the shortest subarray sequentially 
covering the first k- 1 keywords. 

When processing the zth string in the paragraph array, if that string is the /th 
keyword, we update the most recent occurrence of that keyword to i. The shortest 
subarray ending at i which sequentially covers the first j keywords consists of the 
shortest subarray ending at the most recent occurrence of the first j — 1 keywords 
plus the elements from the most recent occurrence of the (j - l)th keyword to i. This 
computation is implemented below. 

public static class Subarray { 

// Represent subarray by starting and ending indices, inclusive. 
public Integer start; 
public Integer end; 

public Subarray(Integer start, Integer end) { 
this.start = start; 
this.end = end; 

} 

1 

public static Subarray findSmallestSequentiallyCoveringSubset( 

List<String> paragraph, List<String> keywords) { 

// Maps each keyword to its index in the keywords array. 

Map<String , Integer> keywordToIdx = new HashMapoQ ; 

// Since keywords are uniquely identified by their indices in keywords 
// array, we can use those indices as keys to lookup in a vector. 

Listdnteger> latestOccurrence = new ArrayList<>(keywords.size()); 

// For each keyword (identified by its index in keywords array), stores the 
// length of the shortest subarray ending at the most recent occurrence of 
// that keyword that sequentially cover all keywords up to that keyword. 

Listdnteger> shortestSubarrayLength = new ArrayList<>(keywords.size()); 

// Initializes latestOccurrence, shortestSubarrayLength, and keywordToIdx. 
for (int i = ®; i < keywords. size () ; ++i) { 
latestOccurrence.add(-l); 

shortestSubarrayLength.add(Integer.MAX_VALUE); 
keywordToIdx.put(keywords.get(i), i); 

} 

int shortestDistance = Integer.MAX_VALUE; 

Subarray result = new Subarray(-1, -1); 
for (int i = ®; i < paragraph. size () ; ++i) { 

Integer keywordldx = keywordToIdx.get(paragraph.get(i)); 
if (keywordldx != null) { 

if (keywordldx == ®) { // First keyword. 

shortestSubarrayLength.set(®, 1); 

} else if (shortestSubarrayLength.get(keywordldx - 1) 
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!= Integer.MAX_VALUE) { 
int distanceToPreviousKeyword 

= i - latestOccurrence.get(keywordldx - 1); 
shortestSubarrayLength.set( 

keywordldx, distanceToPreviousKeyword 

+ shortestSubarrayLength.get(keywordldx - 1)); 

} 

latestOccurrence.set(keywordldx, i); 

// Last keyword, look for improved subarray. 
if (keywordldx == keywords.size() - 1 

&& shortestSubarrayLength.get(shortestSubarrayLength.size() - 1) 

< shortestDistance) { 
shortestDistance 

= shortestSubarrayLength.get(shortestSubarrayLength.size() - 1); 
result.start 
= i 

- shortestSubarrayLength.get(shortestSubarrayLength.size() - 1) 

+ l; 

result.end = i; 

} 

} 

} 

return result; 

} 


Processing each entry of the paragraph array entails a constant number of lookups and 
updates, leading to an 0(n) time complexity, where n is the length of the paragraph 
array. The additional space complexity is dominated by the three hash tables, i.e., 
0(m), where m is the number of keywords. 


13.9 Find the longest subarray with distinct entries 

Write a program that takes an array and returns the length of a longest subarray 
with the property that all its elements are distinct. For example, if the array is 
(/, s, /, e, t, iv, e, n, w, e) then a longest subarray all of whose elements are distinct is 
{s,f,e, t,w). 

Hint: What should you do if the subarray from indices i to j satisfies the property, but the 
subarray from i to j + 1 does not? 

Solution: We begin with a brute-force approach. For each subarray, we test if all its 
elements are distinct using a hash table. The time complexity is 0(n 3 ), where n is the 
array length since there are 0(n 2 ) subarrays, and their average length is 0(n). 

We can improve on the brute-force algorithm by noting that if a subarray con¬ 
tains duplicates, every array containing that subarray will also contain duplicates. 
Therefore, for any given starting index, we can compute the longest subarray starting 
at that index containing no duplicates in time 0(n), since we can incrementally add 
elements to the hash table of elements from the starting index. This leads to an 0(n 2 ) 
algorithm. As soon as we get a duplicate, we cannot find a longer beginning at the 
same initial index that is duplicate-free. 
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We can improve the time complexity by reusing previous computation as we iterate 
through the array. Suppose we know the longest duplicate-free subarray ending at a 
given index. The longest duplicate-free subarray ending at the next index is either the 
previous subarray appended with the element at the next index, if that element does 
not appear in the longest duplicate-free subarray at the current index. Otherwise it 
is the subarray beginning at the most recent occurrence of the element at the next 
index to the next index. To perform this case analysis as we iterate, all we need 
is a hash table storing the most recent occurrence of each element, and the longest 
duplicate-free subarray ending at the current element. 

For the given example, (/, s, /, e, t, w, e, n, w, e), when we process the element at 
index 2, the longest duplicate-free subarray ending at index 1 is from 0 to 1. The hash 
table tells us that the element at index 2, namely /, appears in that subarray, so we 
update the longest subarray ending at index 2 to being from index 1 to 2. Indices 3-5 
introduce fresh elements. Index 6 holds a repeated value, e, which appears within 
the longest subarray ending at index 5; specifically, it appears at index 3. Therefore, 
the longest subarray ending at index 6 to start at index 4. 

public static int longestSubarrayWithDistinctEntries(Listdnteger> A) { 

// Records the most recent occurrences of each entry. 

Mapdnteger, Integer> mostRecentOccurrence = new HashMap<>() ; 
int longestDupFreeSubarrayStartldx = ®, result = ®; 
for (int i = ®; i < A.sizeO; ++i) { 

Integer dupldx = mostRecentOccurrence.put(A.get(i), i); 

// A.get(i) appeared before. Did it appear in the longest current 
// subarray? 
if (dupldx != null) { 

if (dupldx >= longestDupFreeSubarrayStartldx) { 

result = Math.max(result, i - longestDupFreeSubarrayStartldx); 
longestDupFreeSubarrayStartldx = dupldx + 1; 

} 

} 

} 

result = Math.max(result, A.sizeO - longestDupFreeSubarrayStartldx); 
return result; 

} 


The time complexity is 0(n), since we perform a constant number of operations per 
element. 


13.10 Find the length of a longest contained interval 

Write a program which takes as input a set of integers represented by an array, and 
returns the size of a largest subset of integers in the array having the property that if 
two integers are in the subset, then so are all integers between them. For example, if 
the input is (3, -2,7,9,8,1,2,0, -1,5,8), the largest such subset is {-2,-1,0,1,2,3}, so 
you should return 6. 

Hint: Do you really need a total ordering on the input? 
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Solution: The brute-force algorithm is to sort the array and then iterate through it, 
recording for each entry the largest subset with the desired property ending at that 
entry 

On closer inspection we see that sorting is not essential to the functioning of the 
algorithm. We do not need the total ordering—all we care are about is whether the 
integers adjacent to a given value are present. This suggests using a hash table to 
store the entries. Now we iterate over the entries in the array. If an entry e is present 
in the hash table, we compute the largest interval including e such that all values in 
the interval are contained in the hash table. We do this by iteratively searching entries 

in the hash table of the form e + 1, e + 2,..., and e - 1, e - 2,_When we are done, 

to avoid doing duplicated computation we remove all the entries in the computed 
interval from the hash table, since all these entries are in the same largest contained 
interval. 

As a concrete example, consider A = (10,5,3,11,6,100,4). We initialize the hash 
table to {6,10,3,11,5,100,4}. The first entry in A is 10, and we find the largest interval 
contained in A including 10 by expanding from 10 in each direction by doing lookups 
in the hash table. The largest set is {10,11) and is of size 2. This computation updates 
the hash table to {6,3,5,100,4). The next entry in A is 5. Since it is contained in the 
hash table, we know that the largest interval contained in A including 5 has not been 
computed yet. Expanding from 5, we see that 3,4,6 are all in the hash table, and 2 
and 7 are not in the hash table, so the largest set containing 5 is {3,4,5,6), which is 
of size 4. We update the hash table to (100). The three entries after 5, namely 3,11,6 
are not present in the hash table, so we know we have already computed the longest 
intervals in A containing each of these. Then we get to 100, which cannot be extended, 
so the largest set containing it is (100), which is of size 1. We update the hash table 
to {}. Since 4 is not in the hash table, we can skip it. The largest of the three sets is 
{3,4,5,6), i.e., the size of the largest contained interval is 4. 

public static int longestContainedRange (Listdnteger > A) { 

// unprocessedEntries records the existence of each entry in A. 

Set<Integer> unprocessedEntries = new HashSeto(A) ; 

int maxIntervalSize = ®; 

while (!unprocessedEntries.isEmpty()) { 

int a = unprocessedEntries.iterator().next(); 

unprocessedEntries.remove(a); 

// Finds the lower bound of the largest range containing a. 

int lowerBound = a - 1; 

while (unprocessedEntries.contains(lowerBound)) { 
unprocessedEntries.remove(lowerBound); 

--lowerBound; 

} 

// Finds the upper bound of the largest range containing a. 

int upperBound = a + 1; 

while (unprocessedEntries.contains(upperBound)) { 
unprocessedEntries.remove(upperBound); 

++upperBound; 

} 
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maxIntervalSize = Math.max(upperBound - lowerBound - 1, maxIntervalSize); 

} 

return maxIntervalSize; 


The time complexity of this approach is 0(n), where n is the array length, since we 
add and remove array elements in the hash table no more than once. 


13.11 Compute the average of the top three scores 

Student test scores are recorded in a file. Each line consists of a student ID, which is 
an alphanumeric string, and an integer between 0 and 100, inclusive. 

Write a program which takes as input a file containing test scores and returns the 
student who has the maximum score averaged across his or her top three tests. If the 
student has fewer than three test scores, ignore that student. 

Hint: Generalize to computing the top k scores. 

Solution: A straightforward approach would be to find the scores for each student, 
sort those scores, and average the top three. More specifically, we can find the scores 
for each student in a single pass, recording them using a map which takes a student 
id and stores scores for that student in a dynamic array. When all lines have been 
processed, we iterate through the map, sort each corresponding array, and average 
the top three scores for each student, skipping students with fewer than three scores. 
We track the student with the maximum average score as we iterate through the map. 

The time complexity of this approach is 0(n log n), where n is the number of lines 
in the input file. The worst-case corresponds to an input in which there is a single 
student, since we have to sort that student's score array. The space complexity is 
0(n), regardless of the input. 

Storing the entire set of scores for a student is wasteful, since we use only the top 
three scores. We can use a dynamic data structure to track just the top three scores 
seen so far. If three scores already have been seen for a student, and then a score is 
read for that student which is better than the lowest score of these three scores, we 
evict the lowest score, and add the new score. A min-heap is a natural candidate for 
holding the top three scores for each student. Note that scores can be repeated, so we 
need to use a data structure that supports duplicate entries. 

For example, suppose the first three scores seen for Adam are 97,91, and 96, in that 
order. The min-heap for Adam contains 97 after the first of his scores is read, 91,97 
after the second of his scores is read, and 91,96,97 after the third of his scores is read. 
Suppose the next score for Adam in the file is 88. Since 88 is less than 91 we do not 
update his top three scores. Then if the next score for Adam is 97, which is greater 
than 91, we remove the 91 and add 97, updating his top three scores to 96,97,97. 

public static String findStudentWithHighestBestOfThreeScores( 

Iterator<0bject> nameScoreData) { 

Map<String, PriorityQueue<Integer>> studentScores = new HashMap<>(); 
while (nameScoreData.hasNext()) { 


227 



String name = (String)nameScoreData.next(); 

Integer score = (Integer)nameScoreData.next(); 

PriorityQueuednteger> scores = studentScores.get(name); 
if (scores == null) { 

scores = new PriorityQueue<>(); 
studentScores.put(name, scores); 

} 

scores.add(score); 
if (scores.size() > 3) { 

scores.poll(); // Only keep the top 3 scores. 

} 

} 

String topStudent = "no such student"; 
int currentTopThreeScoresSum = ®; 

for (Map . Entry<String , PriorityQueue <Integer» scores : 
studentScores.entrySet()) { 
if (scores.getValue().size() == 3) { 

int currentScoresSum = getTopThreeScoresSum(scores.getValue()); 
if (currentScoresSum > currentTopThreeScoresSum) { 
currentTopThreeScoresSum = currentScoresSum; 
topStudent = scores . getKey () ; 

} 

} 

} 

return topStudent; 

} 

// Returns the sum of top three scores. 

private static int getTopThreeScoresSum(PriorityQueue<Integer> scores) { 
Iteratordnteger > it = scores . iterator () ; 
int result = ®; 
while (it.hasNext()) { 
result += it.next(); 

} 

return result; 

} 

Since we track at most three scores for a student, updating takes constant-time oper¬ 
ation. Therefore, the algorithm spends 0(1) time per test score, yielding an 0(n) time 
bound. The space complexity is 0(m), where m is the number of distinct students. In 
the worst-case, m = 0(n), but in the best-case it can be much better, e.g., if there are 
only few students but lots of test scores. 

13.12 Compute all string decompositions 

This problem is concerned with taking a string (the "sentence" string) and a set 
of strings (the "words"), and finding the substrings of the sentence which are the 
concatenation of all the words (in any order). For example, if the sentence string 
is "amanaplanacanal" and the set of words is {"can", "apl", "ana"}, "aplanacan" is a 
substring of the sentence that is the concatenation of all words. 

Write a program which takes as input a string (the "sentence") and an array of strings 
(the "words"), and returns the starting indices of substrings of the sentence string 
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which are the concatenation of all the strings in the words array. Each string must 
appear exactly once, and their ordering is immaterial. Assume all strings in the words 
array have equal length. It is possible for the words array to contain duplicates. 

Hint: Exploit the fact that the words have the same length. 

Solution: Let's begin by considering the problem of checking whether a string is the 
concatenation strings in words. We can solve this problem recursively—we find a 
string from words that is a prefix of the given string, and recurse with the remaining 
words and the remaining suffix. 

When all strings in words have equal length, say n, only one distinct string in words 
can be a prefix of the given string. So we can directly check the first n characters of 
the string to see if they are in words. If not, the string cannot be the concatenation of 
words. If it is, we remove that string from words and continue with the remainder of 
the string and the remaining words. 

To find substrings in the sentence string that are the concatenation of the strings 
in words, we can use the above process for each index in the sentence as the starting 
index. 

public static List<Integer> f indAHSubstrings (String s, List<String> words) { 
Map<String, Integer> wordToFreq = new HashMap<>(); 
for (String word : words) { 
increment(word, wordToFreq); 

} 

int unitSize = words.get(®).length(); 

List<Integer> result = new ArrayList<>(); 

for (int i = ®; i + unitSize * words.size() <= s.length(); ++i) { 

if (matchAllWordsInDict(s, wordToFreq, i, words . size () , unitSize)) { 
result.add(i); 

> 

} 

return result; 


private static boolean matchAHWordsInDict(String s, 

Map<String, Integer> wordToFreq, 
int start, int numWords, 
int unitSize) { 

Map<String, Integer> currStringToFreq = new HashMap<>(); 
for (int i = ®; i < numWords; ++i) { 

String currWord 

= s.substring(start + i * unitSize, start + (i + 1) * unitSize); 
Integer freq = wordToFreq.get(currWord); 
if (freq == null) { 
return false; 

} 

increment(currWord, currStringToFreq); 
if (currStringToFreq.get(currWord) > freq) { 

// currWord occurs too many times for a match to be possible. 

return false; 

} 

} 

return true; 
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} 

private static void increment(String word, Map<String, Integer> diet) { 

Integer count = diet.get(word); 
if (count == null) { 
count = ®; 

} 

count++; 

diet.put(word, count); 

} 

We analyze the time complexity as follows. Let m be the number of words and n the 
length of each word. Let N be the length of the sentence. For any fixed i, to check if 
the string of length nm starting at an offset of i in the sentence is the concatenation 
of all words has time complexity 0{nm), assuming a hash table is used to store the 
set of words. This implies the overall time complexity is 0(Nnm). In practice, the 
individual checks are likely to be much faster because we can stop as soon as a 
mismatch is detected. 

The problem is made easier, complexity-wise and implementation-wise, by the 
fact that the words are all the same length—it makes testing if a substring equals the 
concatenation of words straightforward. 

13.13 Test the Collatz conjecture 

The Collatz conjecture is the following: Take any natural number. If it is odd, triple 
it and add one; if it is even, halve it. Repeat the process indefinitely. No matter what 
number you begin with, you will eventually arrive at 1. 

As an example, if we start with 11 we get the sequence 11,34,17,52,26,13,40, 
20,10,5,16,8,4,2,1. Despite intense efforts, the Collatz conjecture has not been 
proved or disproved. 

Suppose you were given the task of checking the Collatz conjecture for the first 
billion integers. A direct approach would be to compute the convergence sequence 
for each number in this set. 

Test the Collatz conjecture for the first n positive integers. 

Hint: How would you efficiently check the conjecture for n assuming it holds for all m < n? 

Solution: Often interview questions are open-ended with no definite good solution— 
all you can do is provide a good heuristic and code it well. 

The Collatz hypothesis can fail in two ways—a sequence returns to a previous 
number in the sequence, which implies it will loop forever, or a sequence goes to 
infinity. The latter cannot be tested with a fixed integer word length, so we simply 
flag overflows. 

The general idea is to iterate through all numbers and for each number repeatedly 
apply the rules till you reach 1. Here are some of the ideas that you can try to accelerate 
the check: 

• Reuse computation by storing all the numbers you have already proved to 
converge to 1; that way, as soon as you reach such a number, you can assume it 
would reach 1. 


230 



• To save time, skip even numbers (since they are immediately halved, and the 
resulting number must have already been checked). 

• If you have tested every number up to k, you can stop the chain as soon as you 
reach a number that is less than or equal to k. You do not need to store the 
numbers below k in the hash table. 

• If multiplication and division are expensive, use bit shifting and addition. 

• Partition the search set and use many computers in parallel to explore the 
subsets, as show in Solution 20.9 on Page 386. 

Since the numbers in a sequence may grow beyond 32 bits, you should use 64-bit 
integer and keep testing for overflow; alternately, you can use arbitrary precision 
integers. 

public static boolean testCollatzConjecture (int n) { 

// Stores odd numbers already tested to converge to 1. 

Set<Long> verifiedNumbers = new HashSet<>(); 

// Starts from 3, since hypothesis holds trivially for 1 and 2. 
for (int i = 3; i<=n; i+=2) { 

Set<Long> sequence = new HashSet<>(); 
long testl = i; 
while (testl >= i) { 

if (!sequence.add(testl)) { 

// We previously encountered testl, so the Collatz sequence 
// has fallen into a loop. This disproves the hypothesis, so 
// we short-circuit, returning false. 

return false; 

} 

if ((testl % 2) != ®) { // Odd number 
if (!verifiedNumbers.add(testl)) { 

break; // testl has already been verified to converge to 1. 

} 

long nextTestl = 3 * testl + 1; // Multiply by 3 and add 1. 
if (nextTestl <= testl) { 

throw new ArithmeticException("Collatz sequence overflow for " + i); 

} 

testl = nextTestl; 

} else { 

testl /= 2; // Even number, halve it. 

} 

} 

} 

return true; 


We cannot say much about time complexity beyond the obvious, namely that it is at 
least proportional to n. 


13.14 Implement a hash function for chess 

The state of a game of chess is determined by what piece is present on each square, as 
illustrated in Figure 13.2 on the following page. Each square may be empty, or have 
one of six classes of pieces; each piece may be black or white. Thus flg(l + 6x2)] =4 


231 



bits suffice per square, which means that a total of 64 X 4 = 256 bits can represent the 
state of the chessboard. (The actual state of the game is slightly more complex, as it 
needs to capture which side is to move, castling rights, en passant , etc., but we will 
use the simpler model for this question.) 


8 
7 
6 
5 
4 
3 
2 

1. f3, e5 2. g4, Wh4 1 

Figure 13.2: Chessboard corresponding to the fastest checkmate, Fool’s Mate. 

Chess playing computers need to store sets of states, e.g., to determine if a partic¬ 
ular state has been evaluated before, or is known to be a winning state. To reduce 
storage, it is natural to apply a hash function to the 256 bits of state, and ignore col¬ 
lisions. The hash code can be computed by a conventional hash function for strings. 
However, since the computer repeatedly explores nearby states, it is advantageous 
to consider hash functions that can be efficiently computed based on incremental 
changes to the board. 

Design a hash function for chess game states. Your function should take a state and 
the hash code for that state, and a move, and efficiently compute the hash code for 
the updated state. 

Hint: XOR is associative, commutative, and fast to compute. Additionally, a © a = 0. 

Solution: A straightforward hash function is to treat the board as a sequence of 64 
base 13 digits. There is one digit per square, with the squares numbered from 0 to 
63. Each digit encodes the state of a square: blank, white pawn, white rook,... ,white 
king, black pawn, ..., black king. We use the hash function Q/?', where q is the 
digit in location i, and p is a prime (see on Page 208 for more details). 

Note that this hash function has some ability to be updated incrementally. If, for 
example, a black knight taken by a white bishop the new hash code can be computed 
by subtracting the terms corresponding to the initial location of the knight and bishop, 
and adding a term for a blank at the initial location of the bishop and a term for the 
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bishop at the knight's original position. 

Now we describe a hash function which is much faster to update. It is based on 
creating a random 64-bit integer code for each of the 13 states that each of the 64 
squares can be in. These 13 X 64 = 832 random codes are constants in the program. 
The hash code for the state of the chessboard is the XOR of the code for each location. 
Updates are very fast—for the example above, we XOR the code for black knight on 
i\, white bishop on i 2 , white bishop on i\, and blank on i 2 . 

Incremental updates to the first hash function entail computing terms like p l which 
is more expensive than computing an XOR, which is why the second hash function is 
preferable. The maximum number of word-level XORs performed is 4, for a capture 
or a castling. 

As an example, consider a simpler game played on a 2 X 2 board, with at most 
two pieces, P and 0 present on the board. At most one piece can be present at a 
board position. Denote the board positions by (0,0), (0,1), (1,0), and (1,1). We use 
the following random 7-bit codes for each individual position: 

• For (0,0): (1100111) 2 for blank, (1011000) 2 for P, (1100010) 2 for O. 

• For (0,1): (111U00) 2 for blank, (1000001) 2 for P, (0001111) 2 for Q. 

• For (1,0): (1100101) 2 for blank, (1101101) 2 for P, (0011101) 2 for Q. 

• For (1,1): (0100001) 2 for blank, (0101100) 2 for P, (1001011) 2 for Q. 

Consider the following state: P is present at (0,0) and Q at (1,1), with the remaining 
positions blank. The hash code for this state is (1011000) 2 © (1111100) 2 © (1100101) 2 © 
(1001011) 2 = (0001010) 2 . Now to compute the code for the state where Q moves to 
(0,1), we XOR the code for the current state with (1001011) 2 (removes Q from (1,1)), 
(0100001) 2 (adds blank at (1,1)), (1111100) 2 (removes blank from (0,1)), and (0001111) 2 
(adds Q at (0,1)). Note that, regardless of the size of the board and the number of 
pieces, this approach uses four XORs to get the updated state. 

Variant: How can you include castling rights and en passant information in the state? 
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Chapter 


Sorting 

PROBLEM 14 (Meshing). Two monotone sequences S, T, of lengths n, m, respec¬ 
tively, are stored in two systems ofn(p + 1), m(p + 1) consecutive memory locations, 
respectively: s,s + l,...,s + n(p + 1) - 1 and t, t + 1, . .., t + m(p + 1) - 1. ... If is 
desired to find a monotone permutation R of the sum [S, T], and place it at the locations 
r,r + 1,...,r + (n + m)(p + 1) -1. 

— "Planning And Coding Of Problems For An Electronic Computing Instrument," 
H. H. Goldstine and J. von Neumann, 1948 


Sorting—rearranging a collection of items into increasing or decreasing order—is a 
common problem in computing. Sorting is used to preprocess the collection to make 
searching faster (as we saw with binary search through an array), as well as identify 
items that are similar (e.g., students are sorted on test scores). 

Naive sorting algorithms run in 0(n 2 ) time. A number of sorting algorithms run 
in 0(n log n) time—heapsort, merge sort, and quicksort are examples. Each has its 
advantages and disadvantages: for example, heapsort is in-place but not stable; merge 
sort is stable but not in-place; quicksort runs 0(n 2 ) time in worst-case. (An in-place 
sort is one which uses (9(1) space; a stable sort is one where entries which are equal 
appear in their original order.) 

A well-implemented quicksort is usually the best choice for sorting. We briefly 
outline alternatives that are better in specific circumstances. 

For short arrays, e.g., 10 or fewer elements, insertion sort is easier to code and 
faster than asymptotically superior sorting algorithms. If every element is known to 
be at most k places from its final location, a min-heap can be used to get an 0(n log k) 
algorithm (Solution 11.3 on Page 180). If there are a small number of distinct keys, 
e.g., integers in the range [0..255], counting sort, which records for each element, the 
number of elements less than it, works well. This count can be kept in an array (if 
the largest number is comparable in value to the size of the set being sorted) or a 
BST, where the keys are the numbers and the values are their frequencies. If there 
are many duplicate keys we can add the keys to a BST, with linked lists for elements 
which have the same key; the sorted result can be derived from an in-order traversal 
of the BST. 

Most sorting algorithms are not stable. Merge sort, carefully implemented, can be 
made stable. Another solution is to add the index as an integer rank to the keys to 
break ties. 

Most sorting routines are based on a compare function that takes two items as input 
and returns -1 if the first item is smaller than the second item, 0 if they are equal and 
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1 otherwise. However, it is also possible to use numerical attributes directly, e.g., in 
radix sort. 

The heap data structure is discussed in detail in Chapter 11. Briefly, a max-heap 
(min-heap) stores keys drawn from an ordered set. It supports <9(log n) inserts and 
0(1) time lookup for the maximum (minimum) element; the maximum (minimum) 
key can be deleted in <9(log n) time. Heaps can be helpful in sorting problems, as 
illustrated by Problems 11.1 on Page 177,11.2 on Page 179, and 11.3 on Page 180. 

Sorting boot camp 

It's important to know how to use effectively the sort functionality provided by your 
language's library. Let's say we are given a student class that implements a compare 
method that compares students by name. To sort an array of students by GPA, we 
pass an explicit compare function to the sort function. 

public static class Student implements Comparable<Student> { 
public String name; 
public double gradePointAverage; 

public int compareTo(Student that) { return name.compareTo(that.name); } 

Student(String name, double gradePointAverage) { 
this. name = name; 

this .gradePointAverage = gradePointAverage; 

} 

} 

public static void sortByGPA(List<Student> students) { 

Collections.sort( 

students, Collections.reverseOrder (new Comparator<Student>() { 

©Override 

public int compare(Student a, Student b) { 

return Double.compare(a.gradePointAverage, b.gradePointAverage); 

} 

})); 

} 

public static void sortByName(List<Student> students) { 

Collections.sort(students); 

} 


The time complexity of any reasonable library sort is 0(n log n) for an array with n 
entries. Most library sorting functions are based on quicksort, which has 0(1) space 
complexity. 

Know your sorting libraries 

To sort an array, use Arrays. sort (A), and to sort a list use Collections. sort (list). 

• The time complexity of Arrays. sort (A) is 0(n log n), where n is length of the 
array A. The space complexity is as high as n/2 object references for randomly 
ordered input arrays. For nearly sorted inputs, both time and space complexity 
are much better: approximately n comparisons, and constant space. 

• Arrays. sort (A) operates on arrays of objects that implement the Comparable 
interface. Collections. sort(C) does the same on lists. 


235 



Sorting problems come in two flavors: (1.) use sorting to make subsequent steps 
in an algorithm simpler, and (2.) design a custom sorting routine. For the 

former, it's fine to use a library sort function, possibly with a custom comparator. 
For the latter, use a data structure like a BST, heap, or array indexed by values. 
[Problems 14.4 and 14.7] 

The most natural reason to sort is if the inputs have a natural ordering, and sorting 
can be used as a preprocessing step to speed up searching. [Problem 14.5] 

For specialized input, e.g., a very small range of values, or a small number of 
values, it's possible to sort in 0(n) time rather than 0(n log n) time. [Problems 6.1 
and 14.7] 

It's often the case that sorting can be implemented in less space than required by 
a brute-force approach. [Problem 14.2] 


• Both Arrays, sort (A, cmp) and Collections, sort (C, customCmp) have the 
provision of sorting according to an explicit comparator object, as shown on the 
preceding page. 

• Collections.sort(L) internally proceeds by forming an array A, calling 
Arrays, sort (A) on that array, and then writing the result back into the list 
L. This implies that the time complexity of Collections, sort(L) is the same 
as that of Arrays, sort (A). Because of the copy, the space complexity is al¬ 
ways 0(n). In particular. Collections, sort(L) applied to a LinkedList has 
0(n log n) time complexity. 

14.1 Compute the intersection of two sorted arrays 

A natural implementation for a search engine is to retrieve documents that match the 
set of words in a query by maintaining an inverted index. Each page is assigned an 
integer identifier, its document-lD. An inverted index is a mapping that takes a word 
w and returns a sorted array of page-ids which contain zv —the sort order could be, 
for example, the page rank in descending order. When a query contains multiple 
words, the search engine finds the sorted array for each word and then computes the 
intersection of these arrays—these are the pages containing all the words in the query. 
The most computationally intensive step of doing this is finding the intersection of 
the sorted arrays. 

Write a program which takes as input two sorted arrays, and returns a new array 
containing elements that are present in both of the input arrays. The input arrays 
may have duplicate entries, but the returned array should be free of duplicates. For 
example, the input is (2,3,3,5,5,6,7,7,8,12) and (5,5,6,8,8,9,10,10), your output 
should be (5,6,8). 

Hint: Solve the problem if the input array lengths differ by orders of magnitude. What if they 
are approximately equal? 

Solution: The brute-force algorithm is a "loop join", i.e., traversing through all the 
elements of one array and comparing them to the elements of the other array. Let m 
and n be the lengths of the two input arrays. 
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public static List<Integer> intersectTwoSortedArrays(List<Integer> A, 

List<Integer> B) { 

List<Integer> intersectionAB = new ArrayList<>(); 
for (int i = ®; i < A.sizeQ; ++i) { 

if (i == ® || A.get(i) != A.get(i - 1)) { 
for (Integer b : B) { 

if (A.get(i).equals (b)) { 

intersectionAB.add(A.get(i)); 
break; 

} 

} 

} 

} 

return intersectionAB; 


The brute-force algorithm has 0{mri) time complexity. 

Since both the arrays are sorted, we can make some optimizations. First, we can 
iterate through the first array and use binary search in array to test if the element is 
present in the second array. 

public static List<Integer> intersectTwoSortedArrays(List<Integer> A, 

List<Integer> B) { 

List<Integer> intersectionAB = new ArrayList<>(); 
for (int i = 8; i < A.sizeO; ++i) { 

if ((i == ® | | A.get(i) != A.get(i - 1)) 

&& Collections.binarySearch(B, A.get(i)) >= ®) { 
intersectionAB.add(A.get(i)); 

} 

> 

return intersectionAB; 


The time complexity is 0(m log n), where m is the length of the array being iterated 
over. We can further improve our run time by choosing the shorter array for the outer 
loop since if n is much smaller than m, then n log(m) is much smaller than m log(n). 

This is the best solution if one set is much smaller than the other. However, it 
is not the best when the array lengths are similar because we are not exploiting the 
fact that both arrays are sorted. We can achieve linear runtime by simultaneously 
advancing through the two input arrays in increasing order. At each iteration, if 
the array elements differ, the smaller one can be eliminated. If they are equal, we 
add that value to the intersection and advance both. (We handle duplicates by 
comparing the current element with the previous one.) For example, if the arrays 
are A = (2,3,3,5,7,11) and B = (3,3,7,15,31), then we know by inspecting the first 
element of each that 2 cannot belong to the intersection, so we advance to the second 
element of A. Now we have a common element, 3, which we add to the result, and 
then we advance in both arrays. Now we are at 3 in both arrays, but we know 3 has 
already been added to the result since the previous element in A is also 3. We advance 
in both again without adding to the intersection. Comparing 5 to 7, we can eliminate 
5 and advance to the fourth element in A, which is 7, and equal to the element that B's 
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iterator holds, so it is added to the result. We then eliminate 11, and since no elements 
remain in A, we return (3,7). 


public static List<Integer> intersectTwoSortedArrays(List<Integer> A, 

List<Integer> B) { 

List<Integer> intersectionAB = new ArrayList<>(); 
int i = ®, j = ®; 

while (i < A.size() && j < B.sizeO) { 

if (A.get(i) == B.get(j) && (i == ® || A.get(i) != A.get(i - 1))) { 
intersectionAB.add(A.get(i)); 

++i ; 

++j ; 

} else if (A.get(i) < B.get(j)) { 

++i ; 

} else { // A.get(i) > B.get(j). 

++j; 

} 

} 

return intersectionAB; 


Since we spend (9(1) time per input array element, the time complexity for the entire 
algorithm is 0(m + n). 


14.2 Merge two sorted arrays 

Suppose you are given two sorted arrays of integers. If one array has enough empty 
entries at its end, it can be used to store the combined entries of the two arrays in 
sorted order. For example, consider (5,13,17, and (3,7,11,19), where ^ 

denotes an empty entry. Then the combined sorted entries can be stored in the first 
array as (3,5,7,11,13,17,19, J). 

Write a program which takes as input two sorted arrays of integers, and updates the 
first to the combined entries of the two arrays in sorted order. Assume the first array 
has enough empty entries at its end to hold the result. 

Hint: Avoid repeatedly moving entries. 

Solution: The challenge in this problem lies in writing the result back into the first 
array—if we had a third array to store the result it, we could solve by iterating through 
the two input arrays in tandem, writing the smaller of the entries into the result. The 
time complexity is 0(m + n), where m and n are the number of entries initially in the 
first and second arrays. 

We cannot use the above approach with the first array playing the role of the result 
and still keep the time complexity 0(m + n). The reason is that if an entry in the 
second array is smaller than some entry in the first array, we will have to shift that 
and all subsequent entries in the first array to the right by 1. In the worst-case, each 
entry in the second array is smaller than every entry in the first array, and the time 
complexity is 0(mn). 

We do have spare space at the end of the first array. We take advantage of this by 
filling the first array from its end. The last element in the result will be written to index 
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m + n - 1. For example, if A = (5,13,1and B = (3,7,11,19), then A is 


updated in the following manner: (5,13,17, 

(5,13,17, 13,17,19, „>, (5,13,17,11,13,17,19, „>, 

(5,5,7,11,13,17,19, „>, (3,5,7,11,13,17,19, „>. 


19, ->, (5,13,17, .,.,17,19,.), 
(5,13,7,11,13,17,19, ._.), 


Note that we will never overwrite an entry in the first array that has not already 
been processed. The reason is that even if every entry of the second array is larger 
than each element of the first array, all elements of the second array will fill up indices 
m to m + n - 1 inclusive, which does not conflict with entries stored in the first array. 
This idea is implemented in the program below. Note the resemblance to Solution 7.4 
on Page 99, where we also filled values from the end. 


public static void mergeTwoSortedArrays(Listdnteger> A, int m, 

List<Integer> B, int n) { 
int a = m - 1, b = n - 1, writeldx = m + n - 1; 
while (a >= Q && b >= Q) { 

A.set(writeldx--, A.get(a) > B.get(b) ? A.get(a--) : B.get(b--)); 

} 

while (b >= 0) { 

A.set(writeldx, B.get(b--)); 

} 


The time complexity is 0(m + n). It uses 0(1) additional space. 


14.3 Remove first-name duplicates 

Design an efficient algorithm for removing all first-name duplicates from an array. For 
example, if the input is ((Ian, Botham), (David, Gower), (Ian, Bell), (Ian, Chappell)), 
one result could be ((Ian, Bell), (David, Gower)); ((David, Gower), (Ian, Botham)) 
would also be acceptable. 

Hint: Bring equal items close together. 

Solution: A brute-force approach is to use a hash table. For the names example, we 
would need a hash function and an equals function which use the first name only. We 
first create the hash table and then iterate over it to write entries to the result array. 
The time complexity is 0(n), where n is the number of items. The hash table has a 
worst-case space complexity of 0(n). 

We can avoid the additional space complexity if we can reuse the input array for 
storing the final result. First we sort the array, which brings equal elements together. 
Sorting can be done in 0(n log n) time. The subsequent elimination of duplicates takes 
0(n) time. Note that sorting an array requires that its elements are comparable. 

public static class Name implements Comparable<Name> { 

String firstName; 

String lastName; 

public int compareTo(Name name) { 

int cmpFirst = firstName.compareTo(name.firstName); 
if (cmpFirst != ®) { 
return cmpFirst; 
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} 

return lastName.compareTo(name.lastName); 

} 

} 

public static void eliminateDuplicate(List<Name> A) { 

Collections.sort(A); // Makes identical elements become neighbors. 
int result = Q; 

for (int first = 1; first < A.sizeO; first++) { 

if (! A.get(first ). firstName.equals(A.get(result).firstName) ) { 

A.set(++result, A.get(first)) ; 

} 

} 

// Shrinks array size. 

A. subList(++result, A.size()).clear(); 


The time complexity is 0(n log n) and the space complexity is 0(1). 

14.4 Render a calendar 

Consider the problem of designing an online calendaring application. One compo¬ 
nent of the design is to render the calendar, i.e., display it visually. 

Suppose each day consists of a number of events, where an event is specified as 
a start time and a finish time. Individual events for a day are to be rendered as 
nonoverlapping rectangular regions whose sides are parallel to the X- and Y-axes. 
Let the X-axis correspond to time. If an event starts at time b and ends at time e, the 
upper and lower sides of its corresponding rectangle must be at b and e, respectively. 
Figure 14.1 represents a set of events. 

Suppose the Y-coordinates for each day's events must lie between 0 and L (a pre¬ 
specified constant), and each event's rectangle must have the same "height" (distance 
between the sides parallel to the X-axis). Your task is to compute the maximum height 
an event rectangle can have. In essence, this is equivalent to the following problem. 

[ 

] F4 


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 

Figure 14.1: A set of nine events. The earliest starting event begins at time 1; the latest ending event 

ends at time 17. The maximum number of concurrent events is 3, e.g., {El, £5, £8} as well as others. 




Write a program that takes a set of events, and determines the maximum number of 
events that take place concurrently. 

Hint: Focus on endpoints. 

Solution: The number of events scheduled for a given time changes only at times that 
are start or end times of an event. This leads the following brute-force algorithm. For 
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each endpoint, compute the number of events that contain it. The maximum number 
of concurrent events is the maximum of this quantity over all endpoints. If there are 
n intervals, the total number of endpoints is 2 n. Computing the number of events 
containing an endpoint takes 0(n) time, since checking whether an interval contains 
a point takes 0(1) time. Therefore, the overall time complexity is 0(2n X n) = 0(n 2 ). 

The inefficiency in the brute-force algorithm lies in the fact that it does not take 
advantage of locality, i.e., as we move from one endpoint to another. Intuitively, we 
can improve the run time by sorting the set of all the endpoints in ascending order. (If 
two endpoints have equal times, and one is a start time and the other is an end time, 
the one corresponding to a start time comes first. If both are start or finish times, we 
break ties arbitrarily.) 

Now as we proceed through endpoints we can incrementally track the number of 
events taking place at that endpoint using a counter. For each endpoint that is the 
start of an interval, we increment the counter by 1, and for each endpoint that is the 
end of an interval, we decrement the counter by 1. The maximum value attained by 
the counter is maximum number of overlapping intervals. 

For the example in Figure 14.1 on the preceding page, the first seven endpoints 
are 1 (start), 2(start), 4(start), 5(end), 5(end), 6(start), 7(end). The counter values are up¬ 
dated to 1, 2, 3, 2,1, 2,1. 


public static class Event { 
public int start, finish; 

public Event (int start, int finish) { 
this. start = start; 
this. finish = finish; 

} 


} 

private static class Endpoint implements Comparable<Endpoint> { 
public int time; 
public boolean isStart; 

public int compareTo(Endpoint e) { 
if (time != e.time) { 

return Integer.compare(time, e.time); 

} 

// If times are equal, an endpoint that starts an interval comes first. 
return isStart <&& !e.isStart ? -1 : !isStart && e.isStart ? 1 : Q; 

} 

Endpoint (int t, boolean is) { 
time = t; 
isStart = is; 

} 

} 

public static int findMaxSimultaneousEvents(List<Event> A) { 

// Builds an array of all endpoints . 

List<Endpoint> E = new ArrayList<>(); 
for (Event event : A) { 


241 



E.add(new Endpoint(event.start, true)); 

E.add(new Endpoint(event.finish, false)); 

} 

// Sorts the endpoint array according to the time, breaking ties 
// by putting start times before end times. 

Collections.sort(E); 

// Track the number of simultaneous events, and record the maximum 
// number of simultaneous events. 

int maxNumSimultaneousEvents = ©, numSimultaneousEvents = ©; 
for (Endpoint endpoint : E) { 
if (endpoint.isStart) { 

++numSimultaneousEvents; 
maxNumSimultaneousEvents 

= Math.max(numSimultaneousEvents, maxNumSimultaneousEvents); 
} else { 

--numSimultaneousEvents; 

} 

} 

return maxNumSimultaneousEvents; 


Sorting the endpoint array takes 0(n log n) time; iterating through the sorted array 
takes 0(n) time, yielding an 0(n log n) time complexity. The space complexity is 0(n), 
which is the size of the endpoint array. 

Variant: Users 1,2,... ,n share an Internet connection. User i uses b { bandwidth from 
time Si to fi, inclusive. What is the peak bandwidth usage? 


14.5 Merging intervals 

Suppose the time during the day that a person is busy is stored as a set of disjoint 
time intervals. If an event is added to the person's calendar, the set of busy times may 
need to be updated. 

In the abstract, we want a way to add an interval to a set of disjoint intervals and 
represent the new set as a set of disjoint intervals. For example, if the initial set of 
intervals is [-4, -1], [0,2], [3,6], [7,9], [11,12], [14,17], and the added interval is [1,8], 
the result is [-4, -1], [0,9], [11,12], [14,17]. 

Write a program which takes as input an array of disjoint closed intervals with integer 
endpoints, sorted by increasing order of left endpoint, and an interval to be added, 
and returns the union of the intervals in the array and the added interval. Your result 
should be expressed as a union of disjoint intervals sorted by left endpoint. 

Hint: What is the union of two closed intervals? 

Solution: A brute-force approach is to find the smallest left endpoint and the largest 
right endpoint in the set of intervals in the array and the added interval. We then 
form the result by testing every integer between these two values for membership in 
an interval. The time complexity is 0(Dn), where D is the difference between the two 
extreme values and n is the number of intervals. Note that D may be much larger 
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than n. For example, the brute-force approach will iterate over all integers from 0 to 
1000000 if the array is ([0,1], [999999,1000000]) and the added interval is [10,20]. 

The brute-force approach examines values that are not endpoints, which is waste¬ 
ful, since if an integer point p is not an endpoint, it must lie in the same interval as 
p — 1 does. A better approach is to focus on endpoints, and use the sorted property to 
quickly process intervals in the array. 

Specifically, processing an interval in the array takes place in three stages: 

(1.) First, we iterate through intervals which appear completely before the interval 
to be added—all these intervals are added directly to the result. 

(2.) As soon as we encounter an interval that intersects the interval to be added, 
we compute its union with the interval to be added. This union is itself an 
interval. We iterate through subsequent intervals, as long as they intersect with 
the union we are forming. This single union is added to the result. 

(3.) Finally, we iterate through the remaining intervals. Because the array was 
originally sorted, none of these can intersect with the interval to be added, so 
we add these intervals to the result. 

Suppose the sorted array of intervals is [-4,-1], [0,2], [3,6], [7,9], [11,12], [14,17], 
and the added interval is [1,8]. We begin in Stage 1. Interval [-4,-1] does not 
intersect [1,8], so we add it directly to the result. Next we proceed to [0,2]. Since [0,2] 
intersects [1,8], we are now in Stage 2 of the algorithm. We add the union of the two, 
[0,8], to the result. Now we process [3,6]—it lies completely in [0,8], so we proceed 
to [7,9]. It intersects [1,8] but is not completely contained in it, so we update the most 
recently added interval to the result, [1,8] to [0,9]. Next we proceed to [11,12]. It does 
not intersect the most recently added interval to the result, [0,9], so we are in Stage 3. 
We add it and all subsequent intervals to the result, which is now [-4,-1], [0,9], 
[11,12], [14,17]. Note how the algorithm operates "locally"—sortedness guarantees 
that we do not miss any combinations of intervals. 

The program implementing this idea is given below. 

private static class Interval { 
public int left, right; 

public Interval (int 1, int r) { 
this. left = 1; 
this. right = r; 

} 

} 

public static List<Interval> addlnterval(List<Interval> disjointlntervals, 

Interval newlnterval) { 

int i = Q; 

List dnterval > result = new ArrayList <>() ; 

// Processes intervals in disjointlntervals which come before newlnterval . 
while (i < disjointlntervals.size() 

<&<& newlnterval . left > dis j ointlntervals . get (i) . right) { 
result.add(disj ointlntervals.get(i++)); 

} 

// Processes intervals in disjointlntervals which overlap with newlnterval. 
while (i < disjointlntervals.size() 
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&& newlnterval.right >= disjointlntervals.get(i).left) { 

// If [a, b] and [c, d] overlap, their union is [min(a, c),max(b, d)]. 
newlnterval = new Interval( 

Math.min(newlnterval.left, disjointlntervals.get(i).left), 

Math.max(newlnterval.right, disjointlntervals.get(i).right)); 

++i ; 

} 

result.add(newlnterval); 

// Processes intervals in disjointlntervals which come after newlnterval. 
result.addAll(disj ointlntervals.subList(i, disjointlntervals.size())); 
return result; 


Since the program spends 0( 1) time per entry, its time complexity is 0(n). 


14.6 Compute the union of intervals 

In this problem we consider sets of intervals with integer endpoints; the intervals 
may be open or closed at either end. We want to compute the union of the intervals 
in such sets. A concrete example is given in Figure 14.2. 
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Union of intervals 


Figure 14.2: A set of intervals and their union. 


Design an algorithm that takes as input a set of intervals, and outputs their union 
expressed as a set of disjoint intervals. 

Hint: Do a case analysis. 

Solution: The brute-force approach of considering every number from the minimum 
left endpoint to the maximum right endpoint described at the start of Solution 14.5 
on Page 242 will work for this problem too. As before, its time complexity is 0(Dn), 
where D is the difference between the two extreme values and n is the number of 
intervals, which is unacceptable when D is large. 

We can improve the run time by focusing on intervals instead of individual values. 
We perform the following iteratively: select an interval arbitrarily, and find all inter¬ 
vals it intersects with. If it does not intersect any interval, remove it from the set and 
add it to the result. Otherwise, take its union with all the intervals it intersects with 
(the union must itself be an interval), remove it and all the intervals it intersects with 
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from the result, and add the union to the set. Since we remove at least one interval 
from the set each time, and the time to process each interval (test intersection and 
form unions) is 0(n), the time complexity is 0(n 2 ). 

A faster implementation of the above approach is to process the intervals in sorted 
order, so that we can limit our attention to a subset of intervals as we proceed. 
Specifically, we begin by sorting the intervals on their left endpoints. The idea is 
that this allows us to not have to revisit intervals which are entirely to the left of the 
interval currently being processed. 

We refer to an interval which does not include its left endpoint as being left-open. 
Left-closed, right-open, and right-closed are defined similarly. When sorting, if two 
intervals have the same left endpoint, we put intervals which are left-closed first. We 
break ties arbitrarily. 

As we iterate through the sorted array of intervals, we have the following cases: 

• The interval most recently added to the result does not intersect the current 
interval, nor does its right endpoint equal the left endpoint of the current 
interval. In this case, we simply add the current interval to the end of the result 
array as a new interval. 

• The interval most recently added to the result intersects the current interval. In 
this case, we update the most recently added interval to the union of it with the 
current interval. 

• The interval most recently added to the result has its right endpoint equal to 
the left endpoint of the current interval, and one (or both) of these endpoints 
are closed. In this case too, we update the most recently added interval to the 
union of it with the current interval. 

For the example in Figure 14.2 on the facing page, the result array updates in the fol¬ 
lowing way: <(0,3)), <(0,4]>, <(0,4], [5,7)), <(0,4], [5,8)), <(0,4], [5,11)), <(0,4], [5,11]), 
<(0,4], [5,11], [12,14]), <(0,4], [5,11], [12,16]), <(0,4], [5,11], [12,17)). 

public static class Interval implements Comparablednterval> { 
public Endpoint left = new Endpoint(); 
public Endpoint right = new Endpoint(); 

private static class Endpoint { 
public boolean isClosed; 
public int val; 

} 

public int compareTo(Interval i) { 

if (Integer.compare(left.val, i.left.val) != ®) { 
return left.val - i.left.val; 

} 

// Left endpoints are equal, so now see if one is closed and the 
// other open - closed intervals should appear first. 
if (left.isClosed && !i.left.isClosed) { 
return -1; 

} 

if (!left.isClosed && i.left.isClosed) { 
return 1; 

} 

return ®; 

} 
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©Override 

public boolean equals(Object obj) { 

if (obj == null || !(obj instanceof Interval)) { 

return false; 

} 

if (this == obj) { 
return true; 

} 

Interval that = (Interval)obj; 

return left.val == that.left.val && left.isClosed == that.left.isClosed; 

} 

©Override 

public int hashCode() { return Objects.hash(left.val, left.isClosed); } 

} 

public static Listclnterval> unionOfIntervals(Listdnterval> intervals) { 

// Empty input. 

if (intervals.isEmpty()) { 

return Collections.EMPTY_LIST; 

} 

// Sort intervals according to left endpoints of intervals. 

Collections.sort(intervals); 

Interval curr = intervals.get(©); 

List dnterval > result = new ArrayList <>() ; 
for (int i = 1; i < intervals. size () ; ++i) { 
if (intervals.get(i).left.val < curr.right.val 

|| (intervals.get(i).left.val == curr.right.val 

&& (intervals.get(i).left.isClosed || curr.right.isClosed))) { 
if (intervals.get(i).right.val > curr.right.val 

|| (intervals.get(i).right.val == curr.right.val 
&& intervals.get(i).right.isClosed)) { 
curr.right = intervals.get(i).right; 

} 

} else { 

result.add(curr); 

curr = intervals.get(i); 

} 

} 

result.add(curr); 
return result; 


The time complexity is dominated by the sort step, i.e., 0(n log n). 


14.7 Partitioning and sorting an array with many repeated entries 

Suppose you need to reorder the elements of a very large array so that equal elements 
appear together. For example, if the array is (b, a, c, b, d, a, b, d ) then {a, a, b, b, b, c, d, d) 
is an acceptable reordering, as is (d, d, c, a, a, b, b , b). 

If the entries are integers, this reordering can be achieved by sorting the array 
If the number of distinct integers is very small relative to the size of the array, an 
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efficient approach to sorting the array is to count the number of occurrences of each 
distinct integer and write the appropriate number of each integer, in sorted order, to 
the array. When array entries are objects, with multiple fields, only one of which is to 
be used as a key, the problem is harder to solve. 

You are given an array of student objects. Each student has an integer-valued age field 
that is to be treated as a key. Rearrange the elements of the array so that students of 
equal age appear together. The order in which different ages appear is not important. 
How would your solution change if ages have to appear in sorted order? 

Hint: Count the number of students for each age. 

Solution: The brute-force approach is to sort the array, comparing on age. If the 
array length is n, the time complexity is 0(n log n ) and space complexity is 0(1). The 
inefficiency in this approach stems from the fact that it does more than is required—the 
specification simply asks for students of equal age to be adjacent. 

We use the approach described in the introduction to the problem. However, we 
cannot apply it directly, since we need to write objects, not integers—two students 
may have the same age but still be different. 

For example, consider the array ((Greg, 14), (John, 12), (Andy, 11), (Jim, 13), 
(Phil, 12), (Bob, 13), (Chip, 13), (Tim, 14)). We can iterate through the array and record 
the number of students of each age in a hash. Specifically, keys are ages, and values 
are the corresponding counts. For the given example, on completion of the iteration, 
the hash is (14,2), (12,2), (11,1), (13,3). This tells us that we need to write two students 
of age 14, two students of age 12, one student of age 11 and three students of age 13. 
We can write these students in any order, as long as we keep students of equal age 
adjacent. 

If we had a new array to write to, we can write the two students of age 14 starting 
at index 0, the two students of age 12 starting at index 0 + 2 = 2, the one student of 
age 11 at index 2 + 2 = 4, and the three students of age 13 starting at index 4 + 1 = 5. 
We would iterate through the original array, and write each entry into the new array 
according to these offsets. For example, after the first four iterations, the new array 
would be ((Greg, 14), (John, 12), (Andy, 11), (Jim, 13), ^). 

The time complexity of this approach is 0(n), but it entails 0(n) additional space 
for the result array. We can avoid having to allocate a new array by performing the 
updates in-place. The idea is to maintain a subarray for each of the different types 
of elements. Each subarray marks out entries which have not yet been assigned 
elements of that type. We swap elements across these subarrays to move them to 
their correct position. 

In the program below, we use two hash tables to track the subarrays. One is the 
starting offset of the subarray, the other its size. As soon as the subarray becomes 
empty, we remove it. 

private static class Person { 
public Integer age; 
public String name; 

public Person(Integer k, String n) { 
age = k; 
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name 


= n; 

} 

} 

public static void groupByAge(List<Person> people) { 

Mapdnteger, Integer> ageToCount = new HashMap<>(); 
for (Person p : people) { 

if (ageToCount.containsKey(p.age)) { 

ageToCount.put(p.age, ageToCount.get(p.age) + 1); 

} else { 

ageToCount.put(p.age, 1); 

} 

} 

Mapdnteger, Integer> ageToOffset = new HashMap<>() ; 
int offset = ®; 

for (Map . Entrydnteger , Integer> kc : ageToCount . entrySet () ) { 
ageToOffset. put (kc.getKeyO, offset); 
offset += kc.getValue() ; 


while (!ageToOffset.isEmpty()) { 

Map . Entry dnteger , Integer> from 

= ageToOffset.entrySet().iterator().next(); 

Integer toAge = people.get(from.getValue()).age; 

Integer toValue = ageToOffset.get(toAge) ; 

Collections.swap(people, from.getValue(), toValue); 

// Use ageToCount to see when we are finished with a particular age. 
Integer count = ageToCount.get(toAge) - 1; 
ageToCount.put(toAge, count); 
if (count > ®) { 

ageToOffset.put(toAge, toValue + 1); 

} else { 

ageToOffset.remove(toAge) ; 

} 

} 

} 


The time complexity is 0(ri), since the first pass entails n hash table inserts, and 
the second pass spends <9(1) time to move one element to its proper location. The 
additional space complexity is dictated by the hash table, i.e., 0(m), where m is the 
number of distinct ages. 

If the entries are additionally required to appear sorted by age, we can use a BST- 
based map (Chapter 15) to map ages to counts, since the BST-based map keeps ages 
in sorted order. For our example, the age-count pairs would appear in the order 
(11,1), (12,2), (13,3), (14,2). The time complexity becomes 0(n + m log m), since BST 
insertion takes time <9(log m). Such a sort is often referred to as a counting sort. 


14.8 Team photo day— 1 

You are a photographer for a soccer meet. You will be taking pictures of pairs of 
opposing teams. All teams have the same number of players. A team photo consists 
of a front row of players and a back row of players. A player in the back row must 
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be taller than the player in front of him, as illustrated in Figure 14.3. All players in a 
row must be from the same team. 

Back row } ft ft ft ft 'ft ft ijji 'ft ijji 

Front row ft * ft ft ft ft ft ft ft ft ft 

Figure 14.3: A team photo. Each team has 11 players, and each player in the back row is taller than the 
corresponding player in the front row. 


Design an algorithm that takes as input two teams and the heights of the players in 
the teams and checks if it is possible to place players to take the photo subject to the 
placement constraint. 

Hint: First try some concrete inputs, then make a general conclusion. 

Solution: A brute-force approach is to consider every permutation of one array, and 
compare it against the other array, element by element. Suppose there are n players 
in each team. It takes 0(n\) time to enumerate every possible permutation of a team, 
and testing if a permutation leads to a satisfactory arrangement takes 0(n) time. 
Therefore, the time complexity is 0(n\ X n), clearly unacceptable. 

Intuitively, we should narrow the search by focusing on the hardest to place 
players. Suppose we want to place Team A behind Team B. If A's tallest player is 
not taller than the tallest player in B, then it's not possible to place Team A behind 
Team B and satisfy the placement constraint. Conversely, if Team A's tallest player 
is taller than the tallest player in B, we should place him in front of the tallest player 
in B, since the tallest player in B is the hardest to place. Applying the same logic to 
the remaining players, the second tallest player in A should be taller than the second 
tallest player in B, and so on. 

We can efficiently check whether A's tallest, second tallest, etc. players are each 
taller than B's tallest, second tallest, etc. players by first sorting the arrays of player 
heights. Figure 14.4 shows the teams in Figure 14.3 sorted by their heights. 

Back row i ft ft ft ft ft ft 'ft ijji ijji ijji 

Front row * ft ft ft ft ft ft ft ft ft ft 

Figure 14.4: The teams from Figure 14.3 in sorted order. 

The program below uses this idea to test if a given team can be placed in front of 
another team. 

class Player implements Comparable<Player> { 
public Integer height; 

public Player(Integer h) { height = h; } 

©Override 

public int compareTo(Player that) { 

return Integer.compare(height, that.height); 
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} 

} 

class Team { 

public Team(List<Integer> height) { 

players = new ArrayList<Player>(height.size()); 
for (int i = Q; i < height.size(); ++i) { 
players.add (new Player(height.get(i))); 

} 

} 

// Checks if A can be placed in front of B. 

public static boolean validPlacementExists(Team A, Team B) { 
List<Player> ASorted = A.sortPlayersByHeight(); 

List<Player> BSorted = B.sortPlayersByHeight() ; 

for (int i = Q; i < ASorted. size () <&& i < BSorted. size () ; ++i) { 
if (ASorted.get(i).compareTo(BSorted.get(i)) >= Q) { 

return false; 

} 

} 

return true; 

} 

private List<Player> sortPlayersByHeight() { 

List<Player> sortedPlayers = new ArrayList<Player>(players); 
Collections.sort(sortedPlayers); 
return sortedPlayers; 

} 

private List<Player> players; 


The time complexity is that of sorting, i.e., 0{n log n). 


14.9 Implement a fast sorting algorithm for lists 

Implement a routine which sorts lists efficiently. It should be a stable sort, i.e., the 
relative positions of equal elements must remain unchanged. 

Hint: In what respects are lists superior to arrays? 

Solution: The brute-force approach is to repeatedly delete the smallest element in the 
list and add it to the end of a new list. The time complexity is 0(n 2 ) and the additional 
space complexity is 0(n), where n is the number of nodes in the list. We can refine 
the simple algorithm to run in <9(1) space by reordering the nodes, instead of creating 
new ones. 

public static ListNode<Integer> insertionSort(final ListNodecInteger> L) { 
ListNodecInteger> dummyHead = new ListNodeo (®, L); 

ListNodecInteger> iter = L; 

// The sublist consisting of nodes up to and including iter is sorted in 
// increasing order. We need to ensure that after we move to iter.next 
// this property continues to hold. We do this by swapping iter.next 
// with its predecessors in the list till it’s in the right place. 
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while (iter != null <&& iter.next != null) { 
if (iter.data > iter.next.data) { 

ListNodecInteger> target = iter.next, pre = dummyHead; 
while (pre.next.data < target.data) { 
pre = pre.next; 

} 

ListNodecInteger> temp = pre.next; 
pre.next = target; 
iter.next = target.next; 
target.next = temp; 

} else { 

iter = iter.next; 

} 

} 

return dummyHead.next; 


The time complexity is 0(n 2 ), which corresponds to the case where the list is reverse- 
sorted to begin with. The space complexity is 0(1). 

To improve on runtime, we can gain intuition from considering arrays. Quicksort 
is the best all round sorting algorithm for arrays—it runs in time 0(n log n), and is 
in-place. However, it is not stable. Mergesort applied to arrays is a stable 0(n log n) 
algorithm. However, it is not in-place, since there is no way to merge two sorted 
halves of an array in-place in linear time. 

Unlike arrays, lists can be merged in-place—conceptually, this is because insertion 
into the middle of a list is an 0(1) operation. The following program implements a 
mergesort on lists. We decompose the list into two equal-sized sublists around the 
node in the middle of the list. We find this node by advancing two iterators through 
the list, one twice as fast as the other. When the fast iterator reaches the end of the 
list, the slow iterator is at the middle of the list. We recurse on the sublists, and use 
Solution 8.1 on Page 115 (merge two sorted lists) to combine the sorted sublists. 

public static ListNode<Integer> stableSortList(ListNode<Integer> L) { 

// Base cases: L is empty or a single node, nothing to do. 
if (L == null || L.next == null) { 
return L; 

} 


// Find the midpoint of L using a slow and a fast pointer. 
ListNodecInteger> preSlow = null, slow = L, fast = L; 
while (fast != null && fast.next != null) { 
preSlow = slow; 
fast = fast . next. next; 
slow = slow.next; 

} 

preSlow.next = null; // Splits the list into two equal-sized lists. 


return MergeSortedLists.mergeTwoSortedLists(stableSortList(L), 

stableSortList(slow)); 


} 


The time complexity is the same as that of mergesort, i.e., 0(n log n). Though no 
memory is explicitly allocated, the space complexity is <9(log n). This is the maximum 
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function call stack depth, since each recursive call is with an argument that is half as 
long. 


14.10 Compute a salary threshold 

You are working in the finance office for ABC corporation. ABC needs to cut payroll 
expenses to a specified target. The chief executive officer wants to do this by putting 
a cap on last year's salaries. Every employee who earned more than the cap last year 
will be paid the cap this year; employees who earned no more than the cap will see 
no change in their salary. 

For example, if there were five employees with salaries last year were 
$90, $30, $100, $40, and $20, and the target payroll this year is $210, then 60 is a 
suitable salary cap, since 60 + 30 + 60 + 40 + 20 = 210. 

Design an algorithm for computing the salary cap, given existing salaries and the 
target payroll. 

Hint: How does the payroll vary with the cap? 

Solution: Brute-force is not much use—there are an infinite number of possibilities 
for the cap. 

The cap lies between 0 and the maximum current salary. The payroll increases 
with the cap, which suggests using binary search in this range—if a cap is too high, 
no higher cap will work; the same is true if the cap is too low. 

Suppose there are n employees. Let the array holding salary data be A. The 
payroll, P(c), implied by a cap of c is min(A[i], c). Each step of the binary search 
evaluating P(c) which takes time <9(n). As in Solution 12.5 on Page 195, the number 
of binary search steps depends on the largest salary and the desired accuracy. 

We can use a slightly more analytical method to avoid the need for a specified 
tolerance. The intuition is that as we increase the cap, as long as it does not exceed 
someone's salary, the payroll increases linearly. This suggests iterating through the 
salaries in increasing order. Assume the salaries are given by an array A, which 
is sorted. Suppose the cap for a total payroll of T is known to lie between the kth 
and (k + l)th salaries. We want YJi=o A[z] + (n - k)c to equal = T, which solves to 
c = (T-L k ~lm)l(n-k). 

For the given example, A = (20,30,40,90,100), and T = 210. The payrolls for 
caps equal to the salaries in A are (100,140,170,270,280). Since T = 210 lies between 
170 and 270, the cap lies between the 40 and 90. For any cap c between 40 and 
90, the implied payroll is 20 + 30 + 40 + 2c. We want this to be 210, so we solve 
20 + 30 + 40 + 2c = 210 for c, yielding c = 60. 

public static double findSalaryCap (double targetPayroll, 

List<Double> currentSalaries) { 

Collections.sort(currentSalaries); 
double unadjustedSalarySum = Q; 

for (int i = ®; i < currentSalaries.size(); ++i) { 
final double adjustedSalarySum 

= currentSalaries.get(i) * (currentSalaries.size() - i); 
if (unadjustedSalarySum + adjustedSalarySum >= targetPayroll) { 
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return (targetPayroll - unadjustedSalarySum) 

/ (currentSalaries.size() - i) ; 

} 

unadjustedSalarySum += currentSalaries.get(i); 

} 

// No solution, since targetPayroll > existing payroll. 
return -1.®; 


The most expensive operation for this entire solution is sorting A, hence the run time 
is 0(n log ft). Once we have A sorted, we simply iterate through its entries looking 
for the first entry which implies a payroll that exceeds the target, and then solve for 
the cap using an arithmetical expression. 

If we are given the salary array sorted in advance as well as its prefix sums, then 
for a given value of T, we can use binary search to get the cap in <9(log n) time. 

Variant: Solve the same problem using only 0(1) space. 
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Chapter 


Binary Search Trees 

The number of trees which can be formed with 
n + 1 given knots a, jS, y,... = (n + l) n_1 . 

— “A Theorem on Trees," 
A. Cayley, 1889 


Adding and deleting elements to an array is computationally expensive, when the 
array needs to stay sorted. BSTs are similar to arrays in that the stored values (the 
"keys") are stored in a sorted order. BSTs offer the ability to search for a key as well as 
find the min and max elements, look for the successor or predecessor of a search key 
(which itself need not be present in the BST), and enumerate the keys in a range in 
sorted order. However, unlike with a sorted array, keys can be added to and deleted 
from a BST efficiently. 

A BST is a binary tree as defined in Chapter 10 in which the nodes store keys that 
are comparable, e.g., integers or strings. The keys stored at nodes have to respect the 
BST property—the key stored at a node is greater than or equal to the keys stored at 
the nodes of its left subtree and less than or equal to the keys stored in the nodes of 
its right subtree. Figure 15.1 on the facing page shows a BST whose keys are the first 
16 prime numbers. 

Key lookup, insertion, and deletion take time proportional to the height of the tree, 
which can in worst-case be 0{n), if insertions and deletions are naively implemented. 
However, there are implementations of insert and delete which guarantee that the 
tree has height <9(logn). These require storing and updating additional data at the 
tree nodes. Red-black trees are an example of height-balanced BSTs and are widely 
used in data structure libraries. 

A common mistake with BSTs is that an object that's present in a BST is to be 
updated. The consequence is that a lookup for that object will now fail, even though 
it's still in the BST. When a mutable object that's in a BST is to be updated, first remove 
it from the tree, then update it, then add it back. (As a rule, avoid putting mutable 
objects in a BST.) 

The BST prototype is as follows: 

public static class BSTNode <T> { 
public T data; 

public BSTNode<T> left, right; 

} 
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Figure 15.1: An example of a BST. 


Binary search trees boot camp 

Searching is the single most fundamental application of BSTs. Unlike a hash table, a 
BST offers the ability to find the min and max elements, and find the next largest/next 
smallest element. These operations, along with lookup, delete and find, take time 
<9(log n) for library implementations of BSTs. Both BSTs and hash tables use 0(n) 
space—in practice, a BST uses slightly more space. 

The following program demonstrates how to check if a given value is present in a 
BST. It is a nice illustration of the power of recursion when operating on BSTs. 


public static BSTNode<Integer> searchBST(BSTNode<Integer> tree, 
if (tree == null || tree.data == key) { 
return tree ; 

> 

return key < tree.data ? searchBST(tree.left, key) 

: searchBST(tree.right, key); 


} 


int key) { 


Since the program descends tree with in each step, and spends (9(1) time per level, 
the time complexity is 0(h), where h is the height of the tree. 

Know your binary search tree libraries 

There are two BST-based data structures commonly used in Java— Tree Set and 
TreeMap. The former implements the Set interface, and the latter implements the 
Map interface—refer to 210 for a review of the Set and Map functionality. 

The sortedness of the keys in a BST means that TreeSet and TreeMap have function¬ 
ality beyond that specified by Set and textttMap. First, we describe the functionalities 
added by TreeSet. 

• The iterator returned by iteratorO traverses keys in ascending order. (To 
iterate over keys in descending order, use descendinglteratorO.) 

• first()/last() yield the smallest and largest keys in the tree. 

• lower (12)/higher(3) yield the largest element strictly less than the argumen¬ 
t/smallest element strictly greater than the argument 


255 



With a BST you can iterate through elements in sorted order in time 0(n) (regard- 
less of whether it is balanced). [Problem 15.2] 

Some problems need a combination of a BST and a hashtable. For example, if 
you insert student objects into a BST and entries are ordered by GPA, and then 
a student's GPA needs to be updated and all we have is the student's name and 
new GPA, we cannot find the student by name without a full traversal. However, 
with an additional hash table, we can directly go to the corresponding entry in 
the tree. [Problem 15.8] 

Sometimes, it's necessary to augment a BST, e.g., the number of nodes at a subtree 
in addition to its key, or the range of values sorted in the subtree. [Problem 15.13] 

The BST property is a global property—a binary tree may have the property that 
each node's key is greater than the key at its left child and smaller than the key at 
its right child, but it may not be a BST. [Problem 15.1] 


• floor(4.9)/ceiling(5.7) yield the largest element less than or equal to the 
argument/smallest element greater than or equal to the argument. 

• headSet(lQ), tailSet(5), subSet(l, 12) return viewsof the portion of the keys 
lying in the given range. It's particularly important to note that these operations 
take <9(log n) time, since they are backed by the underlying tree. This implies 
changes to the tree may change the view. It also means that operations like 
size() have 0(n) time complexity. 

As always, there are subtleties, e.g., how does first() handle an empty tree, what 
happens if lower O is called with null, is the range in subSet () inclusive, etc. 

The functionalities added by TreeMap are similar, e.g., floorKey(12) is analogous 
to floor(12), headMap(2Q) is analogous to headSet(20). 

The Tree Set and TreeMap constructors permit for an explicit comparator object 
that is used to order keys, and you should be comfortable with the syntax used to 
specify this object. (See on Page 235 for an example of comparator syntax.) 

15.1 Test if a binary tree satisfies the BST property 

Write a program that takes as input a binary tree and checks if the tree satisfies the 
BST property. 

Hint: Is it correct to check for each node that its key is greater than or equal to the key at its left 
child and less than or equal to the key at its right child? 

Solution: A direct approach, based on the definition of a BST, is to begin with the root, 
and compute the maximum key stored in the root's left subtree, and the minimum 
key in the root's right subtree. We check that the key at the root is greater than or 
equal to the maximum from the left subtree and less than or equal to the minimum 
from the right subtree. If both these checks pass, we recursively check the root's left 
and right subtrees. If either check fails, we return false. 

Computing the minimum key in a binary tree is straightforward: we take the 
minimum of the key stored at its root, the minimum key of the left subtree, and the 
minimum key of the right subtree. The maximum key is computed similarly. Note 
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that the minimum can be in either subtree, since a general binary tree may not satisfy 
the BST property. 

The problem with this approach is that it will repeatedly traverse subtrees. In the 
worst-case, when the tree is a BST and each node's left child is empty, the complexity 
is 0(n 2 ), where n is the number of nodes. The complexity can be improved to 0(n) 
by caching the largest and smallest keys at each node; this requires 0(n) additional 
storage for the cache. 

We now present two approaches which have 0(ri) time complexity and 0(h) addi¬ 
tional space complexity, where h is the height of the tree. 

The first approach is to check constraints on the values for each subtree. The initial 
constraint comes from the root. Every node in its left (right) subtree must have a key 
less than or equal (greater than or equal) to the key at the root. This idea generalizes: 
if all nodes in a tree must have keys in the range [/, u], and the key at the root is w 
(which itself must be between [/, u], otherwise the requirement is violated at the root 
itself), then all keys in the left subtree must be in the range [/, w], and all keys stored 
in the right subtree must be in the range [ w ,«]. 

As a concrete example, when applied to the BST in Figure 15.1 on Page 255, 
the initial range is [-oo,oo]. For the recursive call on the subtree rooted at B, the 
constraint is [- 00 ,19]; the 19 is the upper bound required by A on its left subtree. For 
the recursive call starting at the subtree rooted at F, the constraint is [7,19]. For the 
recursive call starting at the subtree rooted at K, the constraint is [23,43]. The binary 
tree in Figure 10.1 on Page 150 is identified as not being a BST when the recursive call 
reaches C—the constraint is [- 00 ,6], but the key at F is 271, so the tree cannot satisfy 
the BST property. 

public static boolean isBinaryTreeBST(BinaryTreeNode<Integer> tree) { 
return areKeysInRange(tree, Integer.MIN_VALUE, Integer.MAX_VALUE); 

} 

private static boolean areKeysInRange(BinaryTreeNode<Integer> tree, 

Integer lower, Integer upper) { 

if (tree == null) { 
return true; 

} else if (Integer.compare(tree.data, lower) < © 

|| Integer.compare(tree.data, upper) > Q) { 
return false; 

} 

return areKeysInRange(tree.left, lower, tree.data) 

&<& areKeysInRange (tree . right , tree.data, upper); 

} 


Alternatively, we can use the fact that an inorder traversal visits keys in sorted 
order. Furthermore, if an inorder traversal of a binary tree visits keys in sorted order, 
then that binary tree must be a BST. (This follows directly from the definition of a 
BST and the definition of an inorder walk.) Thus we can check the BST property 
by performing an inorder traversal, recording the key stored at the last visited node. 
Each time a new node is visited, its key is compared with the key of the previously 
visited node. If at any step in the walk, the key at the previously visited node is 
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greater than the node currently being visited, we have a violation of the BST property. 

All these approaches explore the left subtree first. Therefore, even if the BST 
property does not hold at a node which is close to the root (e.g., the key stored at the 
right child is less than the key stored at the root), their time complexity is still 0(n). 

We can search for violations of the BST property in a BFS manner, thereby reducing 
the time complexity when the property is violated at a node whose depth is small. 

Specifically, we use a queue, where each queue entry contains a node, as well as 
an upper and a lower bound on the keys stored at the subtree rooted at that node. 
The queue is initialized to the root, with lower bound — oo and upper bound oo. We 
iteratively check the constraint on each node. If it violates the constraint we stop— 
the BST property has been violated. Otherwise, we add its children along with the 
corresponding constraint. 

For the example in Figure 15.1 on Page 255, we initialize the queue with 
(A, [- 00 , 00 ]). Each time we pop a node, we first check the constraint. We pop 
the first entry, (A, [- 00 , 00 ]), and add its children, with the corresponding constraints, 
i.e., (B, [- 00 ,19]) and (I, [19, 00 ]). Next we pop (B, [- 00 ,19]), and add its children, i.e., 
(C, [-oo,7]) and (D, [7,19]). Continuing through the nodes, we check that all nodes 
satisfy their constraints, and thus verify the tree is a BST. 

If the BST property is violated in a subtree consisting of nodes within a particular 
depth, the violation will be discovered without visiting any nodes at a greater depth. 
This is because each time we enqueue an entry, the lower and upper bounds on the 
node's key are the tightest possible. 

public static class QueueEntry { 

public BinaryTreeNode<Integer> treeNode; 
public Integer lowerBound, upperBound; 

public QueueEntry(BinaryTreeNode<Integer> treeNode, Integer lowerBound, 

Integer upperBound) { 
this.treeNode = treeNode; 
this.lowerBound = lowerBound; 
this.upperBound = upperBound; 

} 

} 

public static boolean isBinaryTreeBST(BinaryTreeNode<Integer> tree) { 

Queue<QueueEntry> BFSQueue = new LinkedList<>(); 

BFSQueue.add(new QueueEntry(tree, Integer.MIN_VALUE, Integer.MAX_VALUE)); 
QueueEntry headEntry; 

while ((headEntry = BFSQueue.poll()) != null) { 
if (headEntry.treeNode != null) { 

if (headEntry.treeNode.data < headEntry.lowerBound 

|| headEntry.treeNode.data > headEntry.upperBound) { 
return false; 

} 

BFSQueue.add(new QueueEntry(headEntry.treeNode.left, 

headEntry.lowerBound, 
headEntry.treeNode.data)); 

BFSQueue.add(new QueueEntry(headEntry.treeNode.right, 

headEntry.treeNode.data, 
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headEntry.upperBound)); 


} 

} 

return true; 


15.2 Find the first key greater than a given value in a BST 

Write a program that takes as input a BST and a value, and returns the first key 
that would appear in an inorder traversal which is greater than the input value. For 
example, when applied to the BST in Figure 15.1 on Page 255 you should return 29 
for input 23. 

Hint: Perform binary search, keeping some additional state. 

Solution: We can find the desired node in 0(n) time, where n is the number of nodes 
in the BST, by doing an inorder walk. This approach does not use the BST property. 

A better approach is to use the BST search idiom. We store the best candidate for 
the result and update that candidate as we iteratively descend the tree, eliminating 
subtrees by comparing the keys stored at nodes with the input value. Specifically, 
if the current subtree's root holds a value less than or equal to the input value, we 
search the right subtree. If the current subtree's root stores a key that is greater than 
the input value, we search in the left subtree, updating the candidate to the current 
root. Correctness follows from the fact that whenever we first set the candidate, the 
desired result must be within the tree rooted at that node. 

For example, when searching for the first node whose key is greater than 23 in the 
BST in Figure 15.1 on Page 255, the node sequence is A, I,],K,L. Since L has no left 
child, its key, 29, is the result. 

public static BSTNode<Integer> findFirstGreaterThanK(BSTNode<Integer> tree, 

Integer k) { 

BSTNode<Integer> subtree = tree, firstSoFar = null; 
while (subtree != null) { 
if (subtree.data > k) { 
firstSoFar = subtree; 
subtree = subtree.left; 

} else { // Root and all keys in left-subtree are <= k f so skip them. 
subtree = subtree.right; 

} 

} 

return firstSoFar; 

} 


The time complexity is 0(h), where h is the height of the tree. The space complexity 
is 0(1). 

Variant: Write a program that takes as input a BST and a value, and returns the node 
whose key equals the input value and appears first in an inorder traversal of the BST. 
For example, when applied to the BST in Figure 15.2 on the next page, your program 
should return Node B for 108, Node G for 285, and null for 143. 


259 



108 



Figure 15.2: A BST with duplicate keys. 


15.3 Find the k largest elements in a BST 

A BST is a sorted data structure, which suggests that it should be possible to find the 
k largest keys easily. 

Write a program that takes as input a BST and an integer k, and returns the k largest 
elements in the BST in decreasing order. For example, if the input is the BST in 
Figure 15.1 on Page 255 and k = 3, your program should return (53,47,43). 

Hint: What does an inorder traversal yield? 

Solution: The brute-force approach is to do an inorder traversal, which enumerates 
keys in ascending order, and return the last k visited nodes. A queue is ideal for 
storing visited nodes, since it makes it easy to evict nodes visited more than k steps 
previously. A drawback of this approach is that it potentially processes many nodes 
that cannot possibly be in the result, e.g., if k is small and the left subtree is large. 

A better approach is to begin with the desired nodes, and work backwards. We do 
this by recursing first on the right subtree and then on the left subtree. This amounts 
to a reverse-inorder traversal. For the BST in Figure 15.1 on Page 255, the reverse 
inorder visit sequence is (P, O, J,N, K,M,L, /, A, G,H,F,B,E,C,D). 

As soon as we visit k nodes, we can halt. The code below uses a dynamic array 
to store the desired keys. As soon as the array has k elements, we return. We store 
newer nodes at the end of the array, as per the problem specification. 

To find the five biggest keys in the tree in Figure 15.1 on Page 255, we would 
recurse on A, l, O, P, in that order. Returning from recursive calls, we would visit 
P, O, I, in that order, and add their keys to the result. Then we would recurse on 
J, K, N, in that order. Finally, we would visit N and then K, adding their keys to the 
result. Then we would stop, since we have five keys in the array. 

public static List<Integer> findKLargestlnBST(BSTNode<Integer> tree, int k) { 
Listdnteger> kLargestElements = new ArrayList<>(); 
findKLargestlnBSTHelper(tree, k, kLargestElements); 
return kLargestElements; 

} 

private static void findKLargestlnBSTHelper(BSTNodednteger> tree, int k, 

List dnteger > kLargestElements) { 

// Perform reverse inorder traversal . 

if (tree != null &<& kLargestElements . size () < k) { 
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findKLargestlnBSTHelper(tree.right, k, kLargestElements); 
if (kLargestElements.size () < k) { 
kLargestElements.add(tree.data); 

findKLargestlnBSTHelper(tree.left, k, kLargestElements); 

} 

} 

} 


The time complexity is 0(h + k), which can be much better than performing a con¬ 
ventional inorder walk, e.g., when the tree is balanced and k is small. The complexity 
bound comes from the observation that the number of times the program descends in 
the tree can be at most h more than the number of times it ascends the tree, and each 
ascent happens after we visit a node in the result. After k nodes have been added to 
the result, the program stops. 

15.4 Compute the LCA in a BST 

Since a BST is a specialized binary tree, the notion of lowest common ancestor, as 
expressed in Problem 10.4 on Page 157, holds for BSTs too. 

In general, computing the LCA of two nodes in a BST is no easier than computing 
the LCA in a binary tree, since structurally a binary tree can be viewed as a BST where 
all the keys are equal. However, when the keys are distinct, it is possible to improve 
on the LCA algorithms for binary trees. 

Design an algorithm that takes as input a BST and two nodes, and returns the LCA 
of the two nodes. For example, for the BST in Figure 15.1 on Page 255, and nodes C 
and G, your algorithm should return B. Assume all keys are distinct. Nodes do not 
have references to their parents. 

Hint: Take advantage of the BST property. 

Solution: In Solution 10.3 on Page 156 we presented an algorithm for this problem in 
the context of binary trees. The idea underlying that algorithm was to do a postorder 
traversal—the LCA is the first node visited after the two nodes whose LCA we are to 
compute have been visited. The time complexity was 0(ri), where n is the number of 
nodes in the tree. 

This approach can be improved upon when operating on BSTs with distinct keys. 
Consider the BST in Figure 15.1 on Page 255 and nodes C and G. Since both C and 
G holds keys that are smaller than A's key, their LCA must lie in A's left subtree. 
Examining B, since C's key is less than B's key, and B's key is less than G's key. B 
must be the LCA of C and G. 

Let s and b be the two nodes whose LCA we are to compute, and without loss 
of generality assume the key at s is smaller. (Since the problem specified keys are 
distinct, it cannot be that s and b hold equal keys.) Consider the key stored at the root 
of the BST. There are four possibilities: 

• If the root's key is the same as that stored at s or at b , we are done—the root is 
the LCA. 

• If the key at s is smaller than the key at the root, and the key at b is greater than 
the key at the root, the root is the LCA. 
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• If the keys at s and b are both smaller than that at the root, the LCA must lie in 
the left subtree of the root. 

• If both keys are larger than that at the root, then the LCA must lie in the right 
subtree of the root. 


// Input nodes are not null and the key at s is less than or equal to that at 
// b. 

public static BSTNode<Integer> findLCA(BSTNode<Integer> tree, 

BSTNodednteger> s, 

BSTNode<Integer> b) { 

BSTNode<Integer> p = tree; 

while (p.data < s.data || p.data > b.data) { 

// Keep searching since p is outside of [s, b]. 
while (p.data < s.data) { 

p = p.right; // LCA must be in p’s right child. 

} 

while (p.data > b.data) { 

p = p.left; // LCA must be in p’s left child. 

} 

} 

// Now, s.data >= p.data && p.data <= b.data. 
return p; 

} 


Since we descend one level with each iteration, the time complexity is 0(h), where h 
is the height of the tree. 


15.5 Reconstruct a BST from traversal data 

As discussed in Problem 10.12 on Page 165 there are many different binary trees that 
yield the same sequence of visited nodes in an inorder traversal. This is also true 
for preorder and postorder traversals. Given the sequence of nodes that an inorder 
traversal sequence visits and either of the other two traversal sequences, there exists 
a unique binary tree that yields those sequences. Here we study if it is possible to 
reconstruct the tree with less traversal information when the tree is known to be a 
BST. 

It is critical that the elements stored in the tree be unique. If the root contains key 
v and the tree contains more occurrences of v, we cannot always identify from the 
sequence whether the subsequent vs are in the left subtree or the right subtree. For 
example, for the tree rooted at G in Figure 15.2 on Page 260 the preorder traversal 
sequence is 285,243,285,401. The same preorder traversal sequence is seen if 285 
appears in the left subtree as the right child of the node with key 243 and 401 is at the 
root's right child. 

Suppose you are given the sequence in which keys are visited in an inorder traversal 
of a BST, and all keys are distinct. Can you reconstruct the BST from the sequence? 
If so, write a program to do so. Solve the same problem for preorder and postorder 
traversal sequences. 

Hint: Draw the five BSTs on the keys 1,2,3, and the corresponding traversal orders. 
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Solution: First, with some experimentation, we see the sequence of keys generated 
by an inorder traversal is not enough to reconstruct the tree. For example, the key 
sequence (1,2,3) corresponds to five distinct BSTs as shown in Figure 15.3. 





Figure 15.3: Five distinct BSTs for the traversal sequence (1,2,3). 




However, the story for a preorder sequence is different. As an example, con¬ 
sider the preorder key sequence (43,23,37,29,31,41,47,53). The root must hold 43, 
since it's the first visited node. The left subtree contains keys less than 43, i.e., 
23,37,29,31,41, and the right subtree contains keys greater than 43, i.e., 47,53. Fur¬ 
thermore, (23,37,29,31,41) is exactly the preorder sequence for the left subtree and 
(47,53) is exactly the preorder sequence for the right subtree. We can recursively 
reason that 23 and 47 are the roots of the left and right subtree, and continue to build 
the entire tree, which is exactly the subtree rooted at Node I in Figure 15.1 on Page 255. 

Generalizing, in any preorder traversal sequence, the first key corresponds to the 
root. The subsequence which begins at the second element and ends at the last key 
less than the root, corresponds to the preorder traversal of the root's left subtree. 
The final subsequence, consisting of keys greater than the root corresponds to the 
preorder traversal of the root's right subtree. We recursively reconstruct the BST by 
recursively reconstructing the left and right subtrees from the two subsequences then 
adding them to the root. 


public static BSTNode<Integer> rebuildBSTFromPreorder( 
Listdnteger> preorderSequence) { 
return rebuildBSTFromPreorderHelper(preorderSequence, ®, 

preorderSequence.size()); 


} 


// Builds a BST from preorderSequence.subList(start, end). 
private static BSTNode<Integer> rebuildBSTFromPreorderHelper( 
Listdnteger> preorderSequence, int start, int end) { 
if (start >= end) { 

return null; 

} 

int transitionPoint = start + 1; 
while (transitionPoint < end 

&& Integer.compare(preorderSequence.get(transitionPoint), 
preorderSequence.get(start)) 

< ®) { 

++transitionPoint; 

} 

return new BSTNode<>( 

preorderSequence.get(start), 

rebuildBSTFromPreorderHelper(preorderSequence, start + 1, 

transitionPoint), 
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rebuildBSTFromPreorderHelper(preorderSequence, transitionPoint, end)); 


} 


The worst-case input for this algorithm is the pre-order sequence corresponding to 
a left-skewed tree. The worst-case time complexity satisfies the recurrence W(n) = 
W(n-l)+0(n), which solves to 0(n 2 ). The best-case input is a sequence corresponding 
to a right-skewed tree, and the corresponding time complexity is 0(n). When the 
sequence corresponds to a balanced BST, the time complexity is given by B(n) = 
2B(n/2) + 0(n), which solves to 0(n log n). 

The implementation above potentially iterates over nodes multiple times, which 
is wasteful. A better approach is to reconstruct the left subtree in the same iteration as 
identifying the nodes which lie in it. The code shown below takes this approach. The 
intuition is that we do not want to iterate from first entry after the root to the last entry 
smaller than the root, only to go back and partially repeat this process for the root's 
left subtree. We can avoid repeated passes over nodes by including the range of keys 
we want to reconstruct the subtrees over. For example, looking at the preorder key 
sequence (43,23,37,29,31,41,47,53), instead of recursing on (23,37,29,31,41) (which 
would involve an iteration to get the last element in this sequence). We can directly 
recur on (23,37,29,31,41,47,53), with the constraint that we are building the subtree 
on nodes whose keys are less than 43. 

// Global variable, tracks current subtree. 
private static Integer rootldx; 

public static BSTNode<Integer> rebuildBSTFromPreorder( 

Listdnteger> preorderSequence) { 
rootldx = ®; 

return rebuildBSFromPreorderOnValueRange( 

preorderSequence, Integer.MIN_VALUE, Integer.MAX_VALUE); 

} 

// Builds a BST on the subtree rooted at rootldx from preorderSequence on keys 
// in (lowerBound, upperBound). 

private static BSTNode<Integer> rebuildBSFromPreorderOnValueRange( 

Listdnteger> preorderSequence, Integer lowerBound, Integer upperBound) { 
if (rootldx == preorderSequence.size()) { 

return null; 

} 

Integer root = preorderSequence.get(rootldx); 
if (root < lowerBound || root > upperBound) { 
return null; 

} 

++rootIdx; 

// Note that rebuildBSFromPreorderOnValueRange updates rootldx. So the order 
// of following two calls are critical. 

BSTNodednteger> leftSubtree 

= rebuildBSFromPreorderOnValueRange(preorderSequence, lowerBound, root); 
BSTNode<Integer> rightSubtree 

= rebuildBSFromPreorderOnValueRange(preorderSequence, root, upperBound); 
return new BSTNode<>(root, leftSubtree, rightSubtree); 


264 



The worst-case time complexity is 0(n), since it performs a constant amount of work 
per node. Note the similarity to Solution 25.22 on Page 470. 

A postorder traversal sequence also uniquely specifies the BST, and the algorithm 
for reconstructing the BST is very similar to that for the preorder case. 


15.6 Find the closest entries in three sorted arrays 

Design an algorithm that takes three sorted arrays and returns one entry from each 
such that the minimum interval containing these three entries is as small as possible. 
For example, if the three arrays are (5,10,15), (3,6,9,12,15), and (8,16,24), then 
15,15,16 lie in the smallest possible interval. 

Hint: How would you proceed if you needed to pick three entries in a single sorted array? 

Solution: The brute-force approach is to try all possible triples, e.g., with three nested 
for loops. The length of the minimum interval containing a set of numbers is simply 
the difference of the maximum and the minimum values in the triple. The time 
complexity is 0(lmn), where /, m, n are the lengths of each of the three arrays. 

The brute-force approach does not take advantage of the sortedness of the input 
arrays. For the example in the problem description, the smallest intervals containing 

(5.3.16) and (5,3,24) must be larger than the smallest interval containing (5,3,8) 
(since 8 is the maximum of 5,3,8, and 8 < 16 < 24). 

Let's suppose we begin with the triple consisting of the smallest entries in each 
array. Let s be the minimum value in the triple and t the maximum value in the triple. 
Then the smallest interval with left endpoint s containing elements from each array 
must be [s, t], since the remaining two values are the minimum possible. 

Now remove s from the triple and bring the next smallest element from the array 
it belongs to into the triple. Let s' and t' be the next minimum and maximum values 
in the new triple. Observe [s',f] must be the smallest interval whose left endpoint 
is s': the other two values are the smallest values in the corresponding arrays that 
are greater than or equal to s'. By iteratively examining and removing the smallest 
element from the triple, we compute the minimum interval starting at that element. 
Since the minimum interval containing elements from each array must begin with 
the element of some array, we are guaranteed to encounter the minimum element. 

For example, we begin with (5,3,8). The smallest interval whose left endpoint is 
3 has length 8-3 = 5. The element after 3 is 6, so we continue with the triple (5,6,8). 
The smallest interval whose left endpoint is 5 has length 8-5 = 3. The element 
after 5 is 10, so we continue with the triple (10,6,8). The smallest interval whose left 
endpoint is 6 has length 10-6 = 4. The element after 6 is 9, so we continue with 
the triple (10,9,8). Proceeding in this way, we obtain the triples (10,9,16), (10,12,16), 

(15.12.16) , (15,15,16). Out of all these triples, the one contained in a minimum length 
interval is (15,15,16). 

In the following code, we implement a general purpose function which finds the 
closest entries in k sorted arrays. Since we need to repeatedly insert, delete, find the 
minimum, and find the maximum amongst a collection of k elements, a BST is the 
natural choice. 


265 



public static class ArrayData implements Comparable<ArrayData> { 
public int val; 
public int idx; 

public ArrayData (int idx, int val) { 
this .val = val; 
this. idx = idx; 

} 

©Override 

public int compareTo(ArrayData o) { 

int result = Integer.compare(val, o.val); 
if (result == ®) { 

result = Integer.compare(idx, o.idx); 

} 

return result; 


©Override 

public boolean equals(Object obj) { 

if (obj == null || !(obj instanceof ArrayData)) { 

return false; 

} 

if (this == obj) { 
return true; 

} 

ArrayData that = (ArrayData)obj; 

return this. val == that.val && this. idx == that.idx; 


©Override 

public int hashCode() { return Objects.hash(val, idx); } 


public static int findMinDistanceSortedArrays( 

List<Listdnteger>> sortedArrays) { 

// Indices into each of the arrays. 

Listdnteger> heads = new ArrayList<>(sortedArrays.size()); 
for (Listdnteger> arr : sortedArrays) { 
heads.add(®); 

} 

int result = Integer.MAX_VALUE; 

NavigableSet<ArrayData> currentHeads = new TreeSet<>(); 

// Adds the minimum element of each array in to currentHeads. 
for (int i = ®; i < sortedArrays.size(); ++i) { 

currentHeads.add(new ArrayData(i, sortedArrays.get(i).get(heads.get(i)))); 

} 

while (true) { 

result = Math.min(result, 

currentHeads.last().val - currentHeads.first().val); 
int idxNextMin = currentHeads.first().idx; 

// Return if some array has no remaining elements. 
heads.set(idxNextMin, heads.get(idxNextMin) + 1); 

if (heads.get(idxNextMin) >= sortedArrays.get(idxNextMin).size()) { 


266 



return result; 


} 

currentHeads.pollFirst(); 
currentHeads.add(new ArrayData( 

idxNextMin, sortedArrays.get(idxNextMin).get(heads.get(idxNextMin)))); 

} 

} 


The time complexity is 0(n log k), where n is the total number of elements in the 
k arrays. For the special case k - 3 specified in the problem statement, the time 
complexity is 0(n log 3) = 0(n). 


15.7 Enumerate numbers of the form a + b'J2 

Numbers of the form a + b yfq, where a and b are nonnegative integers, and q is an 
integer which is not the square of another integer, have special properties, e.g., they 
are closed under addition and multiplication. Some of the first few numbers of this 
form are given in Figure 15.4. 

(0+0 V2) (1+0 V2) (0+1V2) (2+0V2) (I+1V2) (O+2V2) (2+1V2) (1+2 V2) (2+2 V2) 

•-■ « » » » » « « > 

0.0 1.0 1.414 2.0 2.414 2.828 3.414 3.828 4.828 

Figure 15.4: Some points of the form a + b V2. (For typographical reasons, this figure does not include 
all numbers of the form a + byjl between 0 and 2 + 2 yfl, e.g., 3 + 0 V2,4 + 0 V2, 0 + 3 V2, 3 +1V2 lie 
in the interval but are not included.) 


Design an algorithm for efficiently computing the k smallest numbers of the form 
a + b V2 for nonnegative integers a and b. 

Hint: Systematically enumerate points. 

Solution: A key fact about V2 is that it is irrational, i.e., it cannot equal to | for any 
integers a, b. This implies that if x + y V2 = x' + y' V2, where x and y are integers, then 
x = x' and y = y' (since otherwise V2 = 2 ^-). 

Here is a brute-force solution. Generate all numbers of the form a + b^[l where a 
and b are integers, 0 < a,b < k - This yields exactly k 2 numbers and the k smallest 
numbers must lie in this collection. We can sort these numbers and return the k 
smallest ones. The time complexity is 0(k 2 log (k 2 )) = O^k 2 log k). 

Intuitively, it is wasteful to generate k 2 numbers, since we only care about a small 
fraction of them. 

We know the smallest number is 0 + 0 V2. The candidates for next smallest number 
are 1 + 0 V2 and 0 + 1 V2. From this, we can deduce the following algorithm. We 
want to maintain a collection of real numbers, initialized to 0 + 0 V2. We perform k 
extractions of the smallest element, call it a + b V2, followed by insertion of (a +1) + b V2 
and a + (b + 1) V2 to the collection. 

The operations on this collection are extract the minimum and insert. Since it is 
possible that the same number may be inserted more than once, we need to ensure 
the collection does not create duplicates when the same item is inserted twice. A 


267 




BST satisfies these operations efficiently, and is used in the implementation below. 
It is initialized to contain 0 + 0 V2. We extract the minimum from the BST, which is 
0 + 0 V2, and insert 1 + 0 V2 and 0 + 1V2 to the BST. We extract the minimum from 
the BST, which is 1 + 0 V2, and insert 2 + 0 V2 and 1 + 1 V2 to the BST, which now 
consists of 0 + 1 V2 = 1.414,2 + 0 V2 = 2,1 + 1 V2 = 2.414. We extract the minimum 
from the BST, which is 0 + 1 V2, and insert 1 + 1 V2 and 0 + 2 V2. The first value is 
already present, so the BST updates to 2 + 0 V2 = 2, 1 + 1 V2 = 2.414,0 + 2 V2 = 2.828. 
(Although it's not apparent from this small example, the values we add back to the 
BST may be smaller than values already present in it, so we really need the BST to 
hold values.) 

public static class ABSqrt2 implements Comparable<ABSqrt2> { 
public int a, b; 
public double val; 

public ABSqrt2 (int a, int b) { 
this. a = a; 
this.b = b; 

val = a + b * Math. sqrt (2) ; 

} 

©Override 

public int compareTo(ABSqrt2 o) { return Double.compare(val, o.val); } 


public static List<ABSqrt2> generateFirstKABSqrt2(int k) { 

SortedSet<ABSqrt2> candidates = new TreeSet<>(); 

// Initial for <9 + <9 * sqrt (2). 
candidates.add(new ABSqrt2(®, ©)); 

List<ABSqrt2 > result = new ArrayList<>(); 
while (result.size () < k) { 

ABSqrt2 nextSmallest = candidates.first() ; 
result.add(nextSmallest); 

// Add the next two numbers derived from nextSmallest. 
candidates.add(new ABSqrt2(nextSmallest.a + 1, nextSmallest.b)); 
candidates.add(new ABSqrt2(nextSmallest.a, nextSmallest.b + 1)); 
candidates.remove(nextSmallest); 

} 

return result; 


In each iteration we perform a deletion and two insertions. There are k such insertions, 
so the time complexity is O(klogk). The space complexity is 0(k), since there are not 
more than 2k insertions. 

Now we describe an 0(n) time solution. It is simple to implement, but is less easy 
to understand than the one based on BST. The idea is that the (n + l)th value will be 
the sum of 1 or V2 with a previous value. We could iterate through all the entries in 
the result and track the smallest such value which is greater than nth value. However, 
this takes time 0{n) to compute the (n + l)th element. 

Intuitively, there is no need to examine all prior values entries when computing 
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(n + l)th value. Let's say we are storing the result in an array A. Then we need to track 
just two entries— i, the smallest index such that A[i] + 1 > A[n - 1], and j, smallest 
index such that A[j] + V2 > A[n - 1]. Clearly, the (n + l)th entry will be the smaller of 
A[i] +1 and A[j] + V2. After obtaining the (n + l)th entry, if it is A[i\ +1, we increment 

i. If it is A[j] + V2, we increment /. If A[i] + 1 equals A[j] + V2, we increment both i 
and j. 

To illustrate, suppose A is initialized to (0), and i and j are 0. The computation 
proceeds as follows: 

1. Since A[0] + 1 = 1 < A[0] + V2 = 1.414, we push 1 into A and increment i. Now 
A = <0,1), i = l,j = 0. 

2. Since A[ 1] + 1 = 2 > A[ 0 ] + a/ 2 = 1.414, we push 1.414 into A and increment j. 
Now A = <0,1,1.414), / = 1,7 = 1. 

3. Since A[ 1] + 1 = 2 < A[l] + V2 = 2.414, we push 2 into A and increment i. Now 
A = <0,1,1.414,2), i = 2,; = 1. 

4. Since A[2] + 1 = 2.414 = A[l] + a/ 2 = 2.414, we push 2.414 into A and increment 
both i and j. Now A = (0,1,1.414,2,2.414), i = 3,j = 2. 

5. Since A[3] + 1 = 3 > A[ 2] + V2 = 2.828, we push 2.828 into A and increment 7 . 
Now A = <0,1,1.414,2,2.828), i = 3,7 = 3. 

6 . Since A[3] + 1 = 3 < A[ 3] + a/ 2 = 3.414, we push 3 into A and increment i. Now 
A = <0,1,1.414,2,2.828,3), i = 4 ,7 = 3. 


public static List<ABSqrt2> generateFirstKABSqrt2 (int k) { 

// Will store the first k numbers of the form a + b sqrt(2). 

List<ABSqrt2> result = new ArrayList<>(); 

result. add(new ABSqrt2(®, ®)); 

int i = ®, j = ®; 

for (int n = 1; n < k; ++n) { 

ABSqrt2 resultlPlusl = new ABSqrt2(result.get(i).a + 1, result.get(i).b); 
ABSqrt2 resultJPlusSqrt2 

= new ABSqrt2(result.get(j).a, result.get(j).b + 1); 
result.add(resultIPlusl.val < resultJPlusSqrt2.val ? resultlPlusl 

: resultJPlusSqrt2); 

if (resultlPlusl.compareTo(result.get(result.size() - 1)) == ®) { 

++i ; 

} 

if (resultJPlusSqrt2.compareTo(result.get(result.size() - 1)) == ®) { 

++j; 

} 

} 

return result; 


Each additional element takes 0(1) time to compute, implying an 0(n) time complex¬ 
ity to compute the first n values of the form a + b V2. 


15.8 The most visited pages problem 

You are given a server log file containing billions of lines. Each line contains a 
number of fields. For this problem, the relevant field is an id denoting the page that 
was accessed. 
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Write a function to read the next line from a log file, and a function to find the k 
most visited pages, where k is an input to the function. Optimize performance for the 
situation where calls to the two functions are interleaved. You can assume the set of 
distinct pages is small enough to fit in RAM. 

As a concrete example, suppose the log file ids appear in the following order: 
g, a, t, t, a, a, a, g, t, c, t, a, t, i.e., there are four pages with ids a, c, g, t. After the first 10 
lines have been read, the most common page is a with a count of 4, and the next most 
common page is t with a count of 3. 

Hint: For each page, count of the number of times it has been visited. 

Solution: A brute-force approach is to have the read function add the page on the 
line to the end of a dynamic array. For the find function, the number of times each 
page has been visited can be obtained by sorting the array, and iterating through it. 
The space complexity is 0(n), where n is the number of lines. 

Alternatively, we can store the page-to-visit-count data in a hash table. The k most 
visited pages are the k pages with the highest counts, and can be computed using the 
algorithm for computing the Arth smallest entry in an array given in Solution 12.8 on 
Page 200. (We would need to reverse the comparator to get the k pages with highest 
counts.) Reading a log file entry takes <9(1) time, regardless of whether we keep pages 
in an array or store (page,visit-count) pairs in a hash table. The time to compute the 
k most visited pages is 0(m), where m is the number of distinct pages processed up 
to that point. 

Intuitively, the brute-force algorithm performs poorly when there are many calls 
to computing the k most visited pages. The reason is that it does not take advantage of 
incrementality—processing a few more lines does not change the page-to-visit-counts 
drastically. 

Height-balanced BSTs are a good choice when performing many incremental up¬ 
dates while preserving sortedness. Adding and removing an entry in a height- 
balanced BST on N nodes takes time <9(logN). Therefore it makes sense to store the 
page-to-visit-counts in a balanced BST. The BST nodes store (page,visit-count) pairs. 
These pairs are ordered by visit-count, with ties broken on page. 

Updating the tree after reading a line from the log file is slightly challenging 
because we cannot easily update the corresponding (page,visit-count). The reason is 
that the BST is ordered by visit-counts, not pages. The solution is to use an additional 
data structure, namely a hash table, which maps pages to (page,visit-count) pairs 
in the BST. If the pair is present, the visit-count in the pair in the BST is updated. 
Note that directly changing the pair does not automatically update the BST it lies in. 
The simplest way to update the BST is to delete the pair from the BST, update the 
pair, and then insert the updated pair back into the BST. (Library implementations of 
height-balanced BSTs ensure that inserts and deletes preserve balance.) 

To find the k most visited pages we find the maximum element in the BST and 
make k- 1 calls to the predecessor function. If the tree is height-balanced, the time 
complexity of k - 1 calls to predecessor is 0(k + log m). For k m this compares very 
favorably with having to iterate through the entire collection lines or pages as we did 
in the brute-force approaches. 
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The time complexity of adding a log file entry is dominated by the BST update, 
which is <9(log ra). This is higher than in the brute-force approach, and is the price we 
pay for fast queries. 

For the given example, after the first four entries have been read, the BST contains 
the following (visit-count, page) pairs (1, a), (1, g), (2, t) in this order, and the hash table 
maps a, g, t to (1, a), (1, g), (2, f), respectively After we read the fifth entry, a, we use the 
hash table to find the corresponding entry (1, a) and update it to (2, a), yielding the tree 
(1, g), (2, a), (2, t). After the first ten entries, the tree consists of (1, c), (2, g), (3, t), (4, a). 
The most visited page at this point is a , with a visit count of 4. 

Variant: Write a program for the same problem with <9(1) time complexity for the 
read next line function, and 0(k) time complexity for the find function. 


15.9 Build a minimum height BST from a sorted array 

Given a sorted array, the number of BSTs that can be built on the entries in the array 
grows enormously with its size. Some of these trees are skewed, and are closer to 
lists; others are more balanced. See Figure 15.3 on Page 263 for an example. 

How would you build a BST of minimum possible height from a sorted array? 

Hint: Which element should be the root? 

Solution: Brute-force is not much help here — enumerating all possible BSTs for the 
given array in search of the minimum height one requires a nontrivial recursion, not 
to mention enormous time complexity. 

Intuitively, to make a minimum height BST, we want the subtrees to be as balanced 
as possible—there's no point in one subtree being shorter than the other, since the 
height is determined by the taller one. More formally, balance can be achieved by 
keeping the number of nodes in both subtrees as close as possible. 

Let n be the length of the array. To achieve optimum balance we can make the 
element in the middle of the array, i.e., the |_f_|th entry, the root, and recursively 
compute minimum height BSTs for the subarrays on either side of this entry. 

As a concrete example, if the array is (2,3,5,7,11,13,17,19,23), the root's key 
will be the middle element, i.e., 11. This implies the left subtree is to be built from 
(2,3,5,7), and the right subtree is to be built from (13,17,19,23). To make both of 
these minimum height, we call the procedure recursively. 

public static BSTNodecInteger> buildMinHeightBSTFromSortedArray( 

Listdnteger> A) { 

return buildMinHeightBSTFromSortedArrayHelper(A, ®, A.sizeO); 

} 

// Build a min-height BST over the entries in A.subList(start, end - 1 ). 
private static BSTNode<Integer> buildMinHeightBSTFromSortedArrayHelper( 
Listdnteger> A, int start, int end) { 
if (start >= end) { 

return null; 

} 

int mid = start + ((end - start) / 2); 
return new BSTNode<>( 
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A.get(mid), buildMinHeightBSTFromSortedArrayHelper(A, start, mid), 
buildMinHeightBSTFromSortedArrayHelper(A, mid + 1, end)); 


The time complexity T(n) satisfies the recurrence T(n) = 2T(n/2) + 0(1), which solves 
to T(n) = 0(n). Another explanation for the time complexity is that we make exactly 
n calls to the recursive function and spend 0(1) within each call. 


15.10 Insertion and deletion in a BST 

A BST is a dynamic data structure—in particular, if implemented carefully, key inser¬ 
tion and deletion can be made very fast. 

Design efficient functions for inserting and removing keys in a BST. Assume that all 
elements in the BST are unique, and that your insertion method must preserve this 
property. 

Hint: Deleting leaves is easy. Pay attention to the children of internal nodes when you delete it. 

Solution: A brute-force approach to insertion might be to traverse the entire tree 
looking for the place to add the new key, and then reconstruct the new tree from 
the new ordering. Deletion can be done in the same way. The time complexity is 
0(n), where n is the number of nodes in the tree. In principle, the space complexity 
can be reduced to 0(h ), where h is the height of the tree, using the techniques in 
Solution 25.23 on Page 472 and 25.22 on Page 470. 

For linear data structures like lists and arrays, we cannot improve upon the 0(n) 
time complexity. However, for BSTs we can use the BST property to quickly find the 
part of the tree to update, and the linked nature of the data structure to efficiently add 
or remove keys. The way to achieve efficiency, both in terms of runtime complexity 
and coding effort, is to minimize the number of links to be updated. 

We begin with insertion. Inserting a value into a tree that is empty is trivial: we 
create a node holding the value to be inserted, setting the node's left and right children 
to empty subtrees. 

For a nonempty tree, we insert by searching for the input value. If it exists, we 
return, since duplicates are disallowed. Otherwise we must have reached an empty 
subtree, so we create a node whose key is the input value, and update the node whose 
child was the empty subtree according to the relative value of that node's key and the 
input value. 

For example, if we were to insert the key 9 to the tree in Figure 15.1 on Page 255, 
we would compare 9 with 19, 7, and 11 in that order. Since 9 < 11, and Node F has 
an empty left subtree, we add a new node containing 9 and set F's left subtree to the 
new node. 

Deletion begins with first identifying the node containing the key to be deleted. 
Suppose the node to be deleted has no children. Then we remove the corresponding 
child field in the parent of the node to be deleted. For example, to delete N from the 
tree in Figure 15.1 on Page 255, we set K's right child to null. If the node to be deleted 
has a single child, we update the parent of the node to be deleted to have that child 
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in place of the node to be deleted. For example, to delete G in the example, we would 
set F's right child to H. 

Now we consider the case where the node to be deleted has two children, e.g.. 
Node A. We can effectively delete the node by replacing its contents with the contents 
of its successor (which must appear in the right subchild), and then deleting the 
successor (which is relatively straightforward since it cannot have a left child). For 
example, to delete A, we would replace its content by 23 ( f's content), and then delete 
/ (by setting the left subchild of J's parent I to the right subchild of /, i.e., K). 

public static class BinarySearchTree { 
private static class TreeNode { 
public Integer data; 
public TreeNode left, right; 

public TreeNode(Integer data, TreeNode left, TreeNode right) { 
this. data = data; 
this. left = left; 
this .right = right; 

} 

} 

private TreeNode root = null; 

public boolean insert(Integer key) { 
if (root == null) { 

root = new TreeNode(key, null, null); 

} else { 

TreeNode curr = root; 

TreeNode parent = curr; 
while (curr != null) { 
parent = curr; 

int cmp = Integer.compare(key, curr.data); 
if (cmp == ®) { 

return false; // key already present, no duplicates to be added. 

} else if (cmp < ®) { 
curr = curr.left; 

} else { // cmp > ©. 
curr = curr.right; 

} 

} 

// Insert key according to key and parent. 
if (Integer.compare(key, parent.data) < ®) { 
parent.left = new TreeNode(key, null, null); 

} else { 

parent.right = new TreeNode(key, null, null); 

} 

} 

return true; 

} 

public boolean delete(Integer key) { 

// Find the node with key. 

TreeNode curr = root, parent = null; 

while (curr != null && Integer.compare(curr.data, key) != ®) { 


273 



parent = curr; 

curr = Integer.compare(key, curr.data) < ® ? curr.left : curr.right; 

} 

if (curr == null) { 

// There’s no node with key in this tree. 

return false; 


TreeNode keyNode = curr; 
if (keyNode.right != null) { 

// Find the minimum of the right subtree. 
TreeNode rKeyNode = keyNode.right; 
TreeNode rParent = keyNode; 
while (rKeyNode.left != null) { 
rParent = rKeyNode; 
rKeyNode = rKeyNode.left; 

} 

keyNode.data = rKeyNode.data; 

// Move links to erase the node. 
if (rParent.left == rKeyNode) { 
rParent.left = rKeyNode.right; 

} else { // rParent.left != rKeyNode. 
rParent.right = rKeyNode.right; 

} 

rKeyNode.right = null; 

} else { 

// Update root link if needed. 
if (root == keyNode) { 
root = keyNode.left; 
keyNode.left = null; 

} else { 

if (parent.left == keyNode) { 
parent.left = keyNode.left; 

} else { 

parent.right = keyNode.left; 

} 

keyNode.left = null; 

} 

} 

return true; 


} 


Both insertion and deletion times are dominated by the time to search for a key and 
to find the minimum element in a subtree. Both of these times are proportional to the 
height of the tree. 

In the worst-case, the BST can grow to be skewed. For example, if the initial tree 
is empty, and n successive insertions are done, where each key inserted is larger than 
the previous one, the height of the resulting tree is n. A red-black tree is a BST with 
fast specialized insertion and deletion routines that keep the tree's height <9(log n). 

Variant: Solve the same problem with the added constraint that you can only change 
links. (In particular, you cannot change the key stored at any node.) 
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15.11 Test if three BST nodes are totally ordered 


Write a program which takes two nodes in a BST and a third node, the "middle" 
node, and determines if one of the two nodes is a proper ancestor and the other a 
proper descendant of the middle. (A proper ancestor of a node is an ancestor that 
is not equal to the node; a proper descendant is defined similarly.) For example, in 
Figure 15.1 on Page 255, if the middle is Node /, your function should return true if 
the two nodes are {A,K} or {J,M|. It should return false if the two nodes are {J, P} or 
{/, iC}. You can assume that all keys are unique. Nodes do not have pointers to their 
parents 

Hint: For what specific arrangements of the three nodes does the check pass? 

Solution: A brute-force approach would be to check if the first node is a proper 
ancestor of the middle and the second node is a proper descendant of the middle. If 
this check returns true, we return true. Otherwise, we return the result of the same 
check, swapping the roles of the first and second nodes. For the BST in Figure 15.1 on 
Page 255, with the two nodes being {L, 1} and middle K, searching for K from L would 
be unsuccessful, but searching for K from I would succeed. We would then search for 
L from K, which would succeed, so we would return true. 

Searching has time complexity 0(h), where h is the height of the tree, since we can 
use the BST property to prune one of the two children at each node. Since we perform 
a maximum of three searches, the total time complexity is 0(h ). 

One disadvantage of trying the two input nodes for being the middle's ancestor 
one-after-another is that even when the three nodes are very close, e.g., if the two 
nodes are [A, /} and middle node is 7 in Figure 15.1 on Page 255, if we begin the search 
for the middle from the lower of the two nodes, e.g., from /, we incur the full 0(h) 
time complexity. 

We can prevent this by performing the searches for the middle from both alter¬ 
natives in an interleaved fashion. If we encounter the middle from one node, we 
subsequently search for the second node from the middle. This way we avoid per¬ 
forming an unsuccessful search on a large subtree. For the example of {A,]} and 
middle I in Figure 15.1 on Page 255, we would search for I from both A and /, stop¬ 
ping as soon as we get to I from A, thereby avoiding a wasteful search from /. (We 
would still have to search for / from I to complete the computation.) 

public static boolean pairIncludesAncestorAndDescendantOfM( 

BSTNode<Integer> possibleAncOrDesc®, BSTNode<Integer> possibleAncOrDesc1, 
BSTNode<Integer> middle) { 

BSTNode<Integer> search® = possibleAncOrDesc®, searchl = possibleAncOrDesc1; 

// Perform interleaved searching from possibleAncOrDescl and 

// possibleAncOrDescl for middle. 

while (search® != possibleAncOrDescl && search® != middle 

&& searchl != possibleAncOrDesc® && searchl != middle 
&& (search® != null || searchl != null)) { 
if (search® != null) { 

search® = search®.data > middle.data ? search®.left : search®.right; 

} 

if (searchl != null) { 
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searchl = searchl.data > middle.data ? searchl.left : searchl.right; 

} 

} 

// If both searches were unsuccessful, or we got from possibleAncOrDesc® 
// to possibleAncOrDescl without seeing middle, or from possibleAncOrDescl 
// to possibleAncOrDesc® without seeing middle, middle cannot lie between 
// possibleAncOrDesc® and possibleAncOrDescl . 

if (search® == possibleAncOrDescl || searchl == possibleAncOrDesc® 

|| (search® != middle && searchl != middle)) { 

return false; 

} 


// If we get here, we already know one of possibleAncOrDesc® or 
// possibleAncOrDescl has a path to middle. Check if middle has a path to 
// possibleAncOrDescl or to possibleAncOrDesc®. 

return search® == middle ? searchTarget(middle, possibleAncOrDescl) 

: searchTarget(middle, possibleAncOrDesc®); 


} 


private static boolean searchTarget(BSTNode<Integer> from, 

BSTNodednteger> target) { 
while (from != null && from != target) { 

from = from.data > target.data ? from.left : from.right; 

} 

return from == target; 

} 


When the middle node does have an ancestor and descendant in the pair, the time 
complexity is 0(d), where d is the difference between the depths of the ancestor and 
descendant. The reason is that the interleaved search will stop when the ancestor 
reaches the middle node, i.e., after 0(d) iterations. The search from the middle node 
to the descendant then takes 0(d) steps to succeed. When the middle node does 
not have an ancestor and descendant in the pair, the time complexity is 0(h ), which 
corresponds to a worst-case search in a BST. 


15.12 The range lookup problem 

Consider the problem of developing a web-service that takes a geographical loca¬ 
tion, and returns the nearest restaurant. The service starts with a set of restaurant 
locations—each location includes X and Y-coordinates. A query consists of a location, 
and should return the nearest restaurant (ties can be broken arbitrarily). 

One approach is to build two BSTs on the restaurant locations: Tx sorted on 
the X coordinates, and Ty sorted on the Y coordinates. A query on location (p,q) 
can be performed by finding all the restaurants whose X coordinate is in the interval 
[p-D,p + D], and all the restaurants whose Y coordinate is in the interval [q - D, q + D], 
taking the intersection of these two sets, and finding the restaurant in the intersection 
which is closest to (p,q). Heuristically, if D is chosen correctly, the subsets are small 
and a brute-force search for the closest point is fast. One approach is to start with a 
small value for D and keep doubling it until the final intersection is nonempty. 
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There are other data structures which are more robust, e.g.. Quadtrees and k-d 
trees, but the approach outlined above works well in practice. 

Write a program that takes as input a BST and an interval and returns the BST keys 
that lie in the interval. For example, for the tree in Figure 15.1 on Page 255, and 
interval [16,31], you should return 17,19,23,29,31. 

Hint: How many edges are traversed when the successor function is repeatedly called m times? 

Solution: A brute-force approach would be to perform a traversal (inorder, postorder, 
or preorder) of the BST and record the keys in the specified interval. The time 
complexity is that of the traversal, i.e., 0(ri), where n is the number of nodes in the 
tree. 

The brute-force approach does not exploit the BST property—it would work un¬ 
changed for an arbitrary binary tree. 

We can use the BST property to prune the traversal as follows: 

• If the root of the tree holds a key that is less than the left endpoint of the interval, 
the left subtree cannot contain any node whose key lies in the interval. 

• If the root of the tree holds a key that is greater than the right endpoint of 
the interval, the right subtree cannot contain any node whose key lies in the 
interval. 

• Otherwise, the root of the tree holds a key that lies within the interval, and it is 
possible for both the left and right subtrees to contain nodes whose keys lie in 
the interval. 

For example, for the tree in Figure 15.1 on Page 255, and interval [16,42], we begin 
the traversal at A, which contains 19. Since 19 lies in [16,42], we explore both of A's 
children, namely B and I. Continuing with B, we see B's key 7 is less than 16, so no 
nodes in B's left subtree can lie in the interval [16,42]. Similarly, when we get to I, 
since 43 > 42, we need not explore I's right subtree. 


private static class Interval { 
public int left, right; 

public Interval (int left, int right) { 
this. left = left; 
this. right = right; 

} 

} 

public static List<Integer> rangeLookupInBST(BSTNodeclnteger> tree, 

Interval interval) { 

Listclnteger> result = new ArrayList<>(); 
rangeLookupInBSTHelper(tree, interval, result); 
return result; 


public static void rangeLookupInBSTHelper(BSTNodecInteger> tree, 

Interval interval, 

Listclnteger> result) { 

if (tree == null) { 
return; 

} 
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if (interval.left <= tree.data && tree.data <= interval.right) { 
// tree.data lies in the interval. 

rangeLookupInBSTHelper(tree.left, interval, result); 
result.add(tree.data); 

rangeLookupInBSTHelper(tree.right, interval, result); 

} else if (interval.left > tree.data) { 

rangeLookupInBSTHelper(tree.right, interval, result); 

} else { // interval.right >= tree.data 

rangeLookupInBSTHelper(tree . left, interval, result); 

} 

} 


The time complexity is tricky to analyze. It makes sense to reason about time 
complexity in terms of the number of keys m that lie in the specified interval. 
We partition the nodes into two categories—those that the program recurses on 
and those that it does not. For our working example, the program recurses on 
A, B, F, G, H, I, J, K, L, M, N. Not all of these have keys in the specified interval, but no 
nodes outside of this set can have keys in the interval. Looking more carefully at the 
nodes we recurse on, we see these nodes can be partitioned into three subsets—nodes 
on the search path to 16, nodes on the search path to 42, and the rest. All nodes in 
the third subset must lie in the result, but some of the nodes in the first two subsets 
may or may not lie in the result. The traversal spends 0(h) time visiting the first two 
subsets, and 0(m) time traversing the third subset—each edge is visited twice, once 
downwards, once upwards. Therefore the total time complexity is 0(m + h), which is 
much better than 0(n) brute-force approach when the tree is balanced, and very few 
keys lie in the specified range. 

Augmented BSTs 

Thus far we have considered BSTs in which each node stores a key, a left child, a 
right child, and, possibly, the parent. Adding fields to the nodes can speed up certain 
queries. As an example, consider the following problem. 

Suppose you needed a data structure that supports efficient insertion, deletion, 
lookup of integer keys, as well as range queries, i.e., determining the number of keys 
that lie in an interval. 

We could use a BST, which has efficient insertion, deletion and lookup. To find 
the number of keys that lie in the interval [U, V], we could search for the first node 
with a key greater than or equal to ii, and then call the successor operation ( 10.10 
on Page 163) until we reach a node whose key is greater than V (or we run out of 
nodes). This has 0(h + m) time complexity, where h is the height of the tree and m 
is the number of nodes with keys within the interval. When m is large, this become 
slow. 

We can do much better by augmenting the BST. Specifically, we add a size field to 
each node, which is the number of nodes in the BST rooted at that node. 

For simplicity, suppose we want to find the number of entries that are less than a 
specified value. As an example, say we want to count the number of keys less than 
40 in the BST in Figure 15.1 on Page 255, and that each node has a size field. Since 
the root A's key, 19, is less than 40, the BST property tells us that all keys in A's left 
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subtree are less than 40. Therefore we can add 7 (which we get from the left child's 
size field) and 1 (for A itself) to the running count, and recurse with A's right child. 

Generalizing, let's say we want to count all entries less than v. We initialize count 
to 0. Since there can be duplicate keys in the tree, we search for the first occurrence 
of v in an inorder traversal using Solution 15.2 on Page 259. (If v is not present, 
we stop when we have determined this.) Each time we take a left child, we leave 
count unchanged; each time we take a right child, we add one plus the size of the 
corresponding left child. If v is present, when we reach the first occurrence of v, we 
add the size of v's left child. The same approach can be used to find the number of 
entries that are greater than v, less than or equal to v, and greater than or equal to v. 

For example, to count the number of less than 40 in the BST in Figure 15.1 on 
Page 255 we would search for 40. Since A's key, 19 is less than 40, we update count to 
7 + 1 = 8 and continue from I. Since J's key, 43 is greater than 40, we move to J's left 
child, /. Since J's key, 23 is less than 40, update count to 8 + 1 = 9, and continue from 
K. Since K's key, 37 is less than 40, we update count to 9 + 2 + 1 = 12, and continue 
from N. Since N's key, 41, is greater than 40, we move to N's left child, which is 
empty. No other keys can be less than 40, so we return count, i.e., 12. Note how we 
avoided exploring A and K's left subtrees. 

The time bound for these computations is 0(h), since the search always descends 
the tree. When m is large, e.g., comparable to the total number of nodes in the tree, 
this approach is much faster than repeated calling successor. 

To compute the number of nodes with keys in the interval [L, If], first compute the 
number of nodes with keys less than L and the number of nodes with keys greater 
than U, and subtract that from the total number of nodes (which is the size stored at 
the root). 

The size field can be updated on insert and delete without changing the 0(h) time 
complexity of both. Essentially, the only nodes whose size field change are those 
on the search path to the added/deleted node. Some conditional checks are needed 
for each such node, but these add constant time per node, leaving the 0(h) time 
complexity unchanged. 

15.13 Add credits 

Consider a server that a large number of clients connect to. Each client is identified by 
a string. Each client has a "credit", which is a nonnegative integer value. The server 
needs to maintain a data structure to which clients can be added, removed, queried, 
or updated. In addition, the server needs to be able to add a specified number of 
credits to all clients simultaneously 

Design a data structure that implements the following methods: 

• Insert: add a client with specified credit, replacing any existing entry for the 
client. 

• Remove: delete the specified client. 

• Lookup: return the number of credits associated with the specified client. 

• Add-to-all: increment the credit count for all current clients by the specified 
amount. 

• Max: return a client with the highest number of credits. 
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Hint: Use additional global state. 

Solution: A hash table is a natural data structure for this application. However, it 
does not support efficient max operations, nor is there an obvious way to perform the 
simultaneous increment, short traversing all entries. A BST does have efficient max 
operation, but it too does not natively support the global increment. 

A general principle when adding behaviors to an object is to wrap the object, 
and add functions in the wrapper, which add behaviors before or after delegating to 
the object. In our context, this suggests storing the clients in a BST, and having the 
wrapper track the total increment amount. 

For example, if we have clients A, B, C, with credits 1,2,3, respectively, and want 
to add 5 credits to each, the wrapper sets the total increment amount to 5. A lookup 
on B then is performed by looking up in the BST, which returns 2, and then adding 
5 before returning. If we want to add 4 more credits to each, we simply update the 
total increment amount to 9. 

One issue to watch out for is what happens to clients inserted after a call to the 
add-to-all function. Continuing with the given example, if we were to now add D 
with a credit of 6, the lookup would return 6 + 9, which is an error. 

The solution is simple—subtract the increment from the credit, i.e., add D with a 
credit of 6 - 9 = -3 to the BST. Now a lookup for D will return -3 + 9, which is the 
correct amount. 

More specifically, the BST keys are credits, and the corresponding values are the 
clients with that credit. This makes for fast max-queries. However, to perform 
lookups and removes by client quickly, the BST by itself is not enough (since it is 
ordered by credit, not client id). We can solve this by maintaining an additional hash 
table in which keys are clients, and values are credits. Lookup is trivial. Removes 
entails a lookup in the hash to get the credit, and then a search into the BST to get the 
set of clients with that credit, and finally a delete on that set. 

public static class ClientsCreditsInfo { 

private int offset = ®; 

private Map<String, Integer> clientToCredit = new HashMap<>(); 

private NavigableMapcInteger, Set<String>> creditToClients 
= new TreeHap<>(); 

public void insert(String clientID, int c) { 
remove(clientID); 

clientToCredit.put(clientID, c - offset); 

Set<String> set = creditToClients.get(c - offset); 
if (set == null) { 

set = new HashSet<>(); 
creditToClients.put(c - offset, set); 

} 

set.add(clientID); 

} 

public boolean remove(String clientID) { 

Integer clientCredit = clientToCredit.get(clientID); 
if (clientCredit != null) { 

creditToClients.get(clientCredit).remove(clientID); 
if (creditToClients . get (clientCredit) . isEmptyO) { 
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creditToClients.remove(clientCredit); 


} 

clientToCredit.remove(clientID); 

return true; 

} 

return false; 

} 

public int lookup(String clientID) { 

Integer clientCredit = clientToCredit.get(clientID); 
return clientCredit == null ? -1 : clientCredit + offset; 


public void addAll(int C) { offset += C; } 

public String max() { 

return creditToClients . isEmptyO 
? "" 

: creditToClients.lastEntry().getValue().iterator().next(); 

} 

} 


The time complexity to insert and remove is dominated by the BST, i.e., <9(logn), 
where n is the number of clients in the data structure. Lookup and add-to-all operate 
only on the hash table, and have <9(1) time complexity. Library BST implementations 
uses caching to perform max in <9(1) time. 
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Chapter 


Recursion 

The power of recursion evidently lies in the possibility of defining an 
infinite set of objects by a finite statement. In the same manner, an 
infinite number of computations can be described by a finite recursive 
program, even if this program contains no explicit repetitions. 

— "Algorithms + Data Structures = Programs," 
N. E. Wirth, 1976 


Recursion is a method where the solution to a problem depends partially on solutions 
to smaller instances of related problems. Two key ingredients to a successful use of 
recursion are identifying the base cases, which are to be solved directly, and ensuring 
progress, that is the recursion converges to the solution. 

A divide-and-conquer algorithm works by repeatedly decomposing a problem 
into two or more smaller independent subproblems of the same kind, until it gets to 
instances that are simple enough to be solved directly. The solutions to the subprob¬ 
lems are then combined to give a solution to the original problem. Merge sort and 
quicksort are classical examples of divide-and-conquer. 

Divide-and-conquer is not synonymous with recursion. In divide-and-conquer, 
the problem is divided into two or more independent smaller problems that are of 
the same type as the original problem. Recursion is more general—there may be a 
single subproblem, e.g., binary search, the subproblems may not be independent, e.g., 
dynamic programming, and they may not be of the same type as the original, e.g., 
regular expression matching. In addition, sometimes to improve runtime, and occa¬ 
sionally to reduce space complexity, a divide-and-conquer algorithm is implemented 
using iteration instead of recursion. 

Recursion boot camp 

The Euclidean algorithm for calculating the greatest common divisor (GCD) of two 
numbers is a classic example of recursion. The central idea is that if y > x, the GCD of x 
and y is the GCD of x and y- x. For example, GCD(156,36) = GCD((156-36) = 120,36). 
By extension, this implies that the GCD of x and y is the GCD of x and y mod x, i.e., 
GCD(156,36) = GCD((156 mod 36) = 12,36) = GCD(12,36 mod 12 = 0) = 12. 

public static long GCD(long x, long y) { return y == ® ? x : GCD(y, x % y); } 


Since with each recursive step one of the arguments is at least halved, it means that 
the time complexity is <9(log max(x, y)). Put another way, the time complexity is 0(n), 
where n is the number of bits needed to represent the inputs. The space complexity 
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is also 0(n), which is the maximum depth of the function call stack. (The program is 
easily converted to one which loops, thereby reducing the space complexity to (9(1).) 


Recursion is especially suitable when the input is expressed using recursive rules. 
[Problem 25.26] 

Recursion is a good choice for search, enumeration, and divide-and-conquer. 

[Problems 16.2,16.9, and 25.27] 

Use recursion as alternative to deeply nested iteration loops. For example, 
recursion is much better when you have an undefined number of levels, such 
as the IP address problem generalized to k substrings. [Problem 7.10] 

If you are asked to remove recursion from a program, consider mimicking call 
stack with the stack data structure. [Problem 25.13] 

Recursion can be easily removed from a tail-recursive program by using a while- 
loop—no stack is needed. (Optimizing compilers do this.) [Problem 5.7] 

If a recursive function may end up being called with the same arguments 
more than once, cache the results—this is the idea behind Dynamic Program¬ 
ming (Chapter 17). 


16.1 The Towers of Hanoi problem 

A peg contains rings in sorted order, with the largest ring being the lowest. You are 
to transfer these rings to another peg, which is initially empty. This is illustrated in 
Figure 16.1. 



PI P2 P3 


PI P2 P3 


(a) Initial configuration. 


(b) Desired configuration. 


Figure 16.1 : Tower of Hanoi with 6 pegs. 


Write a program which prints a sequence of operations that transfers n rings from one 
peg to another. You have a third peg, which is initially empty. The only operation 
you can perform is taking a single ring from the top of one peg and placing it on the 
top of another peg. You must never place a larger ring above a smaller ring. 

Hint: If you know how to transfer the top n - 1 rings, how does that help move the nth ring? 

Solution: The insight to solving this problem can be gained by trying examples. 
The three ring transfer can be achieved by moving the top two rings to the third 
peg, then moving the lowest ring (which is the largest) to the second peg, and then 
transferring the two rings on the third peg to the second peg, using the first peg as the 
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intermediary. To transfer four rings, move the top three rings to the third peg, then 
moving the lowest ring (which is the largest) to the second peg, and then transfer the 
three rings on the third peg to the second peg, using the first peg as an intermediary. 
For both the three ring and four ring transfers, the first and third steps are instances of 
the same problem, which suggests the use of recursion. This approach is illustrated 
in Figure 16.2. Code implementing this idea is given below. 



PI P2 P3 


PI P2 P3 


(a) Move all but the lowest ring from PI to P3 (b) Move the lowest ring from PI to P2. 
using P2 as an intermediary. 



PI P2 P3 


PI P2 P3 


(c) Move the rings to P3 to P2 using PI. 


(d) Solved! 


Figure 16.2: A recursive solution to the Tower of Hanoi for n = 6. 


private static final int NUM_PEGS = 3; 

public static void computeTowerHanoi (int numRings) { 

List<Deque<Integer>> pegs = new ArrayList<>(); 
for (int i = ®; i < NUM.PEGS; i++) { 
pegs .add(new LinkedList<Integer>()); 

} 

// Initialize pegs. 

for (int i = numRings; i >= 1; --i) { 
pegs.get(®).addFirst(i); 

} 

computeTowerHanoiSteps(numRings, pegs, ®, 1, 2); 

} 

private static void computeTowerHanoiSteps (int numRingsToMove, 

List<Deque<Integer>> pegs, 
int fromPeg, int toPeg, 
int usePeg) { 

if (numRingsToMove > ®) { 

computeTowerHanoiSteps(numRingsToMove - 1, pegs, fromPeg, usePeg, toPeg); 
pegs.get(toPeg).addFirst(pegs.get(fromPeg).removeFirst()); 

System.out.println("Move from peg " + fromPeg + " to peg " + toPeg); 
computeTowerHanoiSteps(numRingsToMove - 1, pegs, usePeg, toPeg, fromPeg); 

} 

} 


The number of moves, T(n), satisfies the following recurrence: T(n) = T(n - 1) + 1 + 
T(n- 1) = 1 + 2T(n - 1). The first T(n - 1) corresponds to the transfer of the top n - 1 
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rings from PI to P3, and the second T(rt — 1) corresponds to the transfer from P 3 to 
P2. This recurrence solves to T(n) = 2 n - 1. One way to see this is to "unwrap" the 

recurrence: T(n) = 1 + 2 + 4 H-1- 2 k T(n - k). Printing a single move takes (9(1) time, 

so the time complexity is 0(2 n ). 

Variant: Solve the same problem without using recursion. 

Variant: Find the minimum number of operations subject to the constraint that each 
operation must involve P3. 

Variant: Find the minimum number of operations subject to the constraint that each 
transfer must be from PI to P2, P2 to P3, or P3 to PI. 

Variant: Find the minimum number of operations subject to the constraint that a ring 
can never be transferred directly from PI to P2 (transfers from P2 to PI are allowed). 

Variant: Find the minimum number of operations when the stacking constraint is 
relaxed to the following—the largest ring on a peg must be the lowest ring on the 
peg. (The remaining rings on the peg can be in any order, e.g., it is fine to have the 
second-largest ring above the third-largest ring.) 

Variant: You have 2 n disks of n different sizes, two of each size. You cannot place a 
larger disk on a smaller disk, but can place a disk of equal size on top of the other. 
Compute the minimum number of moves to transfer the 2 n disks from PI to P2. 

Variant: You have 2 n disks which are colored black or white. You cannot place a 
white disk directly on top of a black disk. Compute the minimum number of moves 
to transfer the 2 n disks from PI to P2. 

Variant: Find the minimum number of operations if you have a fourth peg, P4. 


16.2 Generate all nonattacking placements of m-Queens 

A nonattacking placement of queens is one in which no two queens are in the same 
row, column, or diagonal. See Figure 16.3 for an example. 




(a) Solution 1. (b) Solution 2. 

Figure 16.3: The only two ways in which four queens can be placed on a 4 x 4 chessboard. 


Write a program which returns all distinct nonattacking placements of n queens on 
an n X n chessboard, where n is an input to the program. 
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Hint: If the first queen is placed at ( i , j), where can the remaining queens definitely not be placed? 


Solution: A brute-force approach is to consider all possible placements of the n 
queens—there are (" ) possible placements which grows very large with n. 

Since we never would place two queens on the same row, a much faster solution 
is to enumerate placements that use distinct rows. Such a placement cannot lead to 
conflicts on rows, but it may lead to conflicts on columns and diagonals. It can be 
represented by an array of length n, where the zth entry is the location of the queen 
on Row i. 

As an example, if n = 4, begin by placing the first row's queen at Column 0. 
Now we enumerate all placements of the form (0, ^). Placing the second row's 

queen at Column 0 leads to a column conflict, so we skip all placements of the form 
(0,0, ^). Placing the second row's queen at Column 1 leads to a diagonal conflict, 

so we skip all placements of the form (0,1, ^). Now we turn to placements of the 

form (0,2,0, ^). Such placements are conflicting because of the conflict on Column 0. 
Now we turn to placements of the form (0,2,1,^) and (0,2,2,^,). Such placements 
are conflicting because of the diagonal conflict between the queens at Row 1 and 
Column 2 and Row 2 and Column 1, and the column conflict between the queens 
at Row 1 and Column 2 and Row 2 and Column 2, respectively, so we move on to 
(0,2,3, u,), which also violates a diagonal constraint. Now we advance to placements 
of the form (0,3,^,^). Both (0,3,1,^) and (0,3,2,^) lead to conflicts, implying there 
is no nonattacking placement possible with a queen placed at Row 0 and Column 0. 
The first nonattacking placement is (1,3,0,2); the only other nonattacking placement 
is (2,0,3,1). 

public static List<List<Integer» nQueens(int n) { 

List<List<Integer>> result = new ArrayList<>(); 
solveNQueens(n, ®, new ArrayList<Integer>(), result); 
return result; 

} 

private static void solveNQueens (int n, int row, Listdnteger> colPlacement, 

ListcListdnteger>> result) { 

if (row == n) { 

// All queens are legally placed. 

result.add(new ArrayListo(colPlacement)); 

} else { 

for (int col = ®; col < n; ++col) { 
colPlacement.add(col); 
if (isValid(colPlacement)) { 

solveNQueens(n, row + 1, colPlacement, result); 

} 

colPlacement.remove(colPlacement.size() - 1); 

} 

} 

} 

// Test if a newly placed queen will conflict any earlier queens 
// placed before. 

private static boolean isValid(List<Integer> colPlacement) { 
int rowID = colPlacement.size() - 1; 
for (int i = ®; i < rowID; ++i) { 
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int diff = Math.abs(colPlacement.get(i) - colPlacement.get(rowID)); 
if (diff == ® || diff == rowID - i) { 

return false; 

} 

} 

return true; 


The time complexity is lower bounded by the number of nonattacking placements. 
No exact form is known for this quantity as a function of n, but it is conjectured to 
tend to n\/c n , where c « 2.54, which is super-exponential. 

Variant: Compute the number of nonattacking placements of n queens on an n X 
n chessboard. 

Variant: Compute the smallest number of queens that can be placed to attack each 
uncovered square. 

Variant: Compute a placement of 32 knights, or 14 bishops, 16 kings or 8 rooks on an 
8 x8 chessboard in which no two pieces attack each other. 


16.3 Generate permutations 


This problem is concerned with computing all permutations of an array. For example, 
if the array is (2,3,5,7) one output could be (2,3,5,7), (2,3,7,5), (2,5,3,7), (2,5,7,3), 


<2,7,3,5), <2,7,5,3), <3,2,5,7), <3,2,7,5), <3,5,2,7), <3,5,7,2), <3,7,2,5), <3,7,5,2), 
<5,2,3,7), <5,2,7,3), <5,3,2,7), <5,3,7,2), <5,7,2,3), <5,7,2,3), <7,2,3,5), <7,2,5,3), 


<7,3,2,5), <7,3,5,2), <7,5,2,3), <7,5,3,2). (Any other ordering is acceptable too.) 


Write a program which takes as input an array of distinct integers and generates all 
permutations of that array. No permutation of the array may appear more than once. 


Hint: How many possible values are there for the first element? 

Solution: Let the input array be A. Suppose its length is n. A truly brute-force 
approach would be to enumerate all arrays of length n whose entries are from A, and 
check each such array for being a permutation. This enumeration can be performed 
recursively, e.g., enumerate all arrays of length n — 1 whose entries are from A, and 
then for each array, consider the n arrays of length n which is formed by adding a 
single entry to the end of that array. Since the number of possible arrays is n n , the 
time and space complexity are staggering. 

A better approach is to recognize that once a value has been chosen for an entry, 
we do not want to repeat it. Specifically, every permutation of A begins with one of 
A[0], A[l ],..., A[n - 1]. The idea is to generate all permutations that begin with A[0], 
then all permutations that begin with A[ 1], and so on. Computing all permutations 
beginning with A[0] entails computing all permutations of A[1 : n— 1], which suggests 
the use of recursion. To compute all permutations beginning with A [1] we swap A[0] 
with A[ 1] and compute all permutations of the updated A[1 : n - 1]. We then restore 
the original state before embarking on computing all permutations beginning with 
A[ 2], and so on. 
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For example, for the array (7,3,5), we would first generate all permutations start¬ 
ing with 7. This entails generating all permutations of (3,5), which we do by finding 
all permutations of (3,5) beginning with 3. Since (5) is an array of length 1, it has 
a single permutation. This implies (3,5) has a single permutation beginning with 3. 
Next we look for permutations of (3,5) beginning with 5. To do this, we swap 3 and 
5, and find, as before, there is a single permutation of (3,5) beginning with 5, namely, 

(5.3) . Hence, there are two permutations of A beginning with 7, namely (7,3,5) and 

(7.5.3) . We swap 7 with 3 to find all permutations beginning with 3, namely (3,7,5) 
and (3,5,7). The last two permutations we add are (5,3,7) and (5,7,3). In all there 
are six permutations. 

public static List<List<Integer>> permutations(List<Integer> A) { 

List<Listclnteger>> result = new ArrayList<>(); 
directedPermutations(®, A, result); 
return result; 

} 

private static void directedPermutations (int i, Listclnteger> A, 

List<List<Integer>> result) { 

if (i == A. size() - 1) { 

result. add(new ArrayList<>(A)); 

return; 

} 

// Try every possibility for A[i]. 
for (int j = i; j < A.sizeO; ++j) { 

Collections.swap(A, i, j); 

// Generate all permutations for A. subList (i + 1, A.sizeO) • 
directedPermutations(i + 1, A, result); 

Collections.swap(A, i, j); 

} 

} 


The time complexity is determined by the number of recursive calls, since within each 
function the time spent is 0( 1), not including the time in the subcalls. The number 
of function calls, C(n) satisfies the recurrence C(n) = 1 + nC(n - 1) for n > 1, with 

C(0) = 1. Expanding this, we see C(n) = 1 + n + n(n - 1) + n(n - 1 )(n - 2) H-+ n\ - 

n\(l/n\ + l/(n - 1)! + l/(n - 2)! H-h 1/1!). The sum (1 + 1/1! + 1/2! H-h 1/n!) tends 

to Euler's number e, so C(n) tends to (e - 1 )n\, i.e., 0(n\). The time complexity T(n) is 
0(n X ft!), since we do 0(n) computation per call outside of the recursive calls. 

Now we describe a qualitatively different algorithm for this problem. In Solu¬ 
tion 6.10 on Page 76 we showed how to efficiently compute the permutation that 
follows a given permutation, e.g., (2,3,1,4) is followed by (2,3,4,1). We can extend 
that algorithm to solve the current problem. The idea is that the n distinct entries in 
the array can be mapped to 1,2,3,..., with 1 corresponding to the smallest entry. For 
example, if the array is (7,3,5), we first sort it to obtain, (3,5,7). Using the approach 
of Solution 6.10 on Page 76, the next array will be (3,7,5), followed by (5,3,7), (5,7,3), 
(7,3,5), (7,5,3). This is the approach in the program below. 

public static ListcListcInteger» permutations(List<Integer> A) { 

List<Listclnteger>> result = new ArrayList<>(); 
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// Generate the first permutation in dictionary order. 
Collections.sort(A) ; 
do { 

result. add(new ArrayList<>(A)) ; 

A = NextPermutation.nextPermutation(A); 

} while ( !A.isEmpty()); 
return result; 


The time complexity is 0(n X nl), since there are n\ permutations and we spend 0(n) 
time to store each one. 


Variant: Solve Problem 16.3 on Page 287 when the input array may have duplicates. 
You should not repeat any permutations. For example, if A = (2,2,3,0) then the out¬ 
put should be <2,2,0,3), <2,2,3,0), <2,0,2,3), <2,0,3,2), (2,3,2,0), <2,3,0,2>, <0,2,2,3), 
<0,2,3,2), <0,3,2,2), <3,2,2,0), <3,2,0,2>, <3,0,2,2>. 


16.4 Generate the power set 

The power set of a set S is the set of all subsets of S, including both the empty set 0 
and S itself. The power set of {0,1,2} is graphically illustrated in Figure 16.4. 



Figure 16.4: The power set of {0,1,2} is {0,{O},{1},{2},{0,1}, {1,2}, {0,2}, {0,1,2}}. 


Write a function that takes as input a set and returns its power set. 

Hint: There are 2" subsets for a given set S of size n. There are 2 k k -bit words. 

Solution: A brute-force way is to compute all subsets U that do not include a partic¬ 
ular element (which could be any single element). Then we compute all subsets V 
which do include that element. Each subset set must appear in U or in V, so the final 
result is just UuV. The construction is recursive, and the base case is when the input 
set is empty, in which case we return {{)). 

As an example, let S = {0,1,2}. Pick any element, e.g., 0. First, we recursively 
compute all subsets of {1,2}. To do this, we select 1. This leaves us with {2}. Now 
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we pick 2, and get to a base case. So the set of subsets of {2} is {} union with {2}, 
i.e., {{},{2}}. The set of subsets of {1,2} then is {{},{2}} union with {{1},{1,2}}, i.e., 
{{}, {2}, {1}, {1,2}}. The set of subsets of {0,1,2} then is {{}, {2}, {1}, {1,2}} union with 
{{0}, {0,2}, {0,1}, {0,1,2}}, i.e., {{}, {2}, {1}, {1,2}, {0}, {0,2}, {0,1}, {0,1,2}}, 

public static ListcListcInteger» generatePowerSet(List<Integer> inputSet) { 

List<Listdnteger>> powerSet = new ArrayList<>(); 

directedPowerSet(inputSet, ®, new ArrayListdnteger>(), powerSet); 
return powerSet; 


// Generate all subsets whose intersection with inputSet[9], 

// inputSet[toBeSelected - 1] is exactly selectedSoFar . 

private static void directedPowerSet(List<Integer> inputSet, int toBeSelected, 

List<Integer> selectedSoFar, 

ListcListdnteger» powerSet) { 
if (toBeSelected == inputSet.size()) { 

powerSet .add(new ArrayListo(selectedSoFar)) ; 
return; 

} 

// Generate all subsets that contain inputSet[toBeSelected]. 
selectedSoFar.add(inputSet.get(toBeSelected)); 

directedPowerSet(inputSet, toBeSelected + 1, selectedSoFar, powerSet); 

// Generate all subsets that do not contain inputSet[toBeSelected]. 
selectedSoFar.remove(selectedSoFar.size() - 1); 

directedPowerSet(inputSet, toBeSelected + 1, selectedSoFar, powerSet); 


The number of recursive calls, C(n) satisfies the recurrence C(n) = 2C(n - 1), which 
solves to C(n) = <9(2"). Since we spend 0(n) time within a call, the time complexity 
is 0(n2 n ). The space complexity is <9(w2"), since there are 2" subsets, and the average 
subset size is nj 2. If we just want to print the subsets, rather than returning all of 
them, we simply perform a print instead of adding the subset to the result, which 
reduces the space complexity to 0(n) —the time complexity remains the same. 

For a given ordering of the elements of S, there exists a one-to-one correspondence 
between the 2" bit arrays of length n and the set of all subsets of S—the Is in the 
n- length bit array v indicate the elements of S in the subset corresponding to v. For 
example, if S = {a,b,c,d}, the bit array (1,0,1,1) denotes the subset \a,c,d\. This 
observation can be used to derive a nonrecursive algorithm for enumerating subsets. 

In particular, when n is less than or equal to the width of an integer on the architec¬ 
ture (or language) we are working on, we can enumerate bit arrays by enumerating 
integers in [0,2" - 1] and examining the indices of bits set in these integers. These in¬ 
dices are determined by first isolating the lowest set bit by computing y = x&~(x -1), 
which is described on Page 25, and then getting the index by computing lg y. 

private static final double L0G_2 = Math.log(2); 

public static ListcListcInteger>> generatePowerSet(Listdnteger> inputSet) { 

ListcListdnteger>> powerSet = new ArrayListc>(); 
for (int intForSubset = ®; intForSubset c (1 cc inputSet.size()); 
++intForSubset) { 
int bitArray = intForSubset; 

List dnteger > subset = new ArrayList c>() ; 
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while (bitArray ! = ®) { 
subset.add( 

inputSet.get( (int) (Math.log(bitArray & ~(bitArray - 1)) / L0G_2))); 
bitArray <&= bitArray - 1; 

} 

powerSet.add(subset); 

} 

return powerSet; 


Since each set takes 0(n) time to compute, the time complexity is 0(n2 n ). In practice, 
this approach is very fast. Furthermore, its space complexity is 0(n) when we want 
to just enumerate subsets, e.g., to print them, rather that to return all the subsets. 

Variant: Solve this problem when the input array may have duplicates, i.e., de¬ 
notes a multiset. You should not repeat any multiset. For example, if A = 
<1,2,3,2>, then you should return (<),<1>,<2),<3),<1,2>, <1,3>,<2,2),<2,3),<1,2,2), 
<1,2,3>, <2,2,3), <1,2,2,3>). 

16.5 Generate all subsets of size k 

There are a number of testing applications in which it is required to compute all 
subsets of a given size for a specified set. 

Write a program which computes all size k subsets of {1,2,..., n\, where k and n are 
program inputs. For example, if k = 2 and n = 5, then the result is the following: 
{{1,2}, {1,3}, {1,4}, {1,5), {2,3}, {2,4), {2,5}, {3,4}, {3,5}, {4,5}} 

Hint: Think of the right function signature. 

Solution: One brute-force approach is to compute all subsets of {1,2,..., n\, and then 
restrict the result to subsets of size k. A convenient aspect of this approach is that 
we can use Solution 16.4 on Page 289 to compute all subsets. The time complexity is 
0(n2 n ), regardless of k. When k is much smaller than n, or nearly equal to n, it ends 
up computing many subsets which cannot possibly be of the right size. 

To gain efficiency, we use a more focused approach. In particular, we can make 
nice use of case analysis. There are two possibilities for a subset—it does not contain 
1, or it does contain 1. In the first case, we return all subsets of size k of {2,3,..., n}; in 
the second case, we compute all A: — 1 sized subsets of {2,3,..., n) and add 1 to each 
of them. 

For example, if n = 4 and k = 2, then we compute all subsets of size 2 from {2,3,4}, 
and all subsets of size 1 from {2,3,4}. We add 1 to each of the latter, and the result is 
the union of the two sets of subsets, i.e., {{2,3}, {2,4}, {3,4}} U {{1,2}, {1,3}, {1,4}}. 

public static List<List<Integer» combinations (int n, int k) { 

List<Listdnteger>> result = new ArrayList<>() ; 

directedCombinations(n, k, 1 , new ArrayList<Integer>() , result); 
return result; 

} 

private static void directedCombinations(int n, int k, int offset, 
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List<Integer> partialCombination, 
List<List<Integer>> result) { 


if (partialCombination.size () == k) { 

result.add (new ArrayList <> (partialCombination) ); 

return; 

} 

// Generate remaining combinations over {offset, n - 1} of size 

// numRemaining. 

final int numRemaining = k - partialCombination.size (); 
for (int i = offset; i <= n && numRemaining <= n - i + 1; ++i) { 
partialCombination.add(i); 

directedCombinations(n, k, i + 1, partialCombination, result); 
partialCombination.remove(partialCombination.size() - 1); 

} 


The time complexity is 0(n( n k )); the reasoning is analogous to that for the recursive 
solution enumerating the powerset (Page 290). 


16.6 Generate strings of matched parens 

Strings in which parens are matched are defined by the following three rules: 

• The empty string, is a string in which parens are matched. 

• The addition of a leading left parens and a trailing right parens to a string in 
which parens are matched results in a string in which parens are matched. For 
example, since "(())()" is a string with matched parens, so is "((())())". 

• The concatenation of two strings in which parens are matched is itself a string 
in which parens are matched. For example, since "(())()" and "()" are strings 
with matched parens, so is "(())()()". 

For example, the set of strings containing two pairs of matched parens 
is {(()),()()}, and the set of strings with three pairs of matched parens is 
{((()))/(()())/( 0 ) 0 / 0 ( 0 ), 0001 - 

Write a program that takes as input a number and returns all the strings with that 
number of matched pairs of parens. 

Hint: Think about what the prefix of a string of matched parens must look like. 

Solution: A brute-force approach would be to enumerate all strings on 2k parenthe¬ 
ses. To test if the parens in a string are matched, we use Solution 9.3 on Page 137, 
specialized to one type of parentheses. There are 2 2k possible strings, which is a lower 
bound on the time complexity. Even if we restrict the enumeration to strings with an 
equal number of left and right parens, there are ( 2k ) strings to consider. 

We can greatly improve upon the time complexity by enumerating in a more 
directed fashion. For example, some strings can never be completed to a string with 
k pairs of matched parens, e.g., if a string begins with ). Therefore, one way to be 
more directed is to build strings incrementally. We will ensure that as each additional 
character is added, the resulting string has the potential to be completed to a string 
with k pairs of matched parens. 
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Suppose we have a string whose length is less than 2k, and we know that string 
can be completed to a string with k pairs of matched parens. How can we extend that 
string with an additional character so that the resulting string can still be completed 
to a string with k pairs of matched parens? 

There are two possibilities: we add a left parens, or we add a right parens. 

• If we add a left parens, and still want to complete the string to a string with k 
pairs of matched parens, it must be that the number of left parens we need is 
greater than 0. 

• If we add a right parens, and still want to complete the string to a string with 
k pairs of matched parens, it must be that the number of left parens we need is 
less than the number of right parens (i.e., there are unmatched left parens in the 
string). 

As a concrete example, if k = 2, we would go through the following sequence 
of strings: "(()", "(())", "0", "()(", "()()"• Of these, "(())" and "()()" are 

complete, and we would add them to the result. 

public static List<String> generateBalancedParentheses (int numPairs) { 
List<String> result = new ArrayList<>(); 

directedGenerateBalancedParentheses(numPairs, numPairs, result); 

return result; 


private static void directedGenerateBalancedParentheses( 

int numLeftParensNeeded, int numRightParensNeeded, String validPrefix, 
List<String> result) { 

if (numLeftParensNeeded == ® && numRightParensNeeded == ®) { 
result.add(validPrefix); 
return; 

> 

if (numLeftParensNeeded > ®) { // Able to insert ’(’. 

directedGenerateBalancedParentheses(numLeftParensNeeded - 1, 

numRightParensNeeded, 
validPrefix + "(", result); 

} 

if (numLeftParensNeeded < numRightParensNeeded) { // Able to insert 
directedGenerateBalancedParentheses(numLeftParensNeeded, 

numRightParensNeeded - 1, 
validPrefix + ")"» result); 

} 

} 


The number C(k) of strings with k pairs of matched parens grows very rapidly with 
k. Specifically, it can be shown that C(k + 1) = E,=o0)/(^ + 1)/ which solves to 
(2k)\/((k\(k +1)!). 

16.7 Generate palindromic decompositions 

A string is said to be palindromic if it reads the same backwards and forwards. A 
decomposition of a string is a set of strings whose concatenation is the string. For 
example, "611116" is palindromic, and "611", "11", "6" is one decomposition for it. 
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Compute all palindromic decompositions of a given string. For example, if the string 
is "0204451881", then the decomposition "020", "44", "5", "1881" is palindromic, as 
is "020", "44", "5", "1", "88", "1". However, "02044, "5", "1881" is not a palindromic 
decomposition. 

Hint: Focus on the first palindromic string in a palindromic decomposition. 

Solution: We can brute-force compute all palindromic decompositions by first com¬ 
puting all decompositions, and then checking which ones are palindromic. To com¬ 
pute all decompositions, we use prefixes of length 1,2... for the first string in the 
decomposition, and recursively compute the decomposition of the corresponding 
suffix. The number of such decompositions is 2 M_1 , where n is the length of the string. 
(One way to understand this is from the fact that every n-bit vector corresponds 
to a unique decomposition—the Is in the bit vector denote the starting point of a 
substring.) 

Clearly, the brute-force approach is inefficient because it continues with decompo¬ 
sitions that cannot possibly be palindromic, e.g., it will recursively compute decom¬ 
positions that begin with "02" for "0204451881". We need a more directed approach— 
specifically, we should enumerate decompositions that begin with a palindrome. 

For the given example, "0204451881", we would recursively compute palindromic 
sequences for "204451881" (since "0" is a palindrome), and for "4451881" (since 
"020" is a palindrome). To compute palindromic decompositions for "204451881", 
we would recursively compute palindromic sequences for "04451881" (since "2" is 
the only prefix that is a palindrome). To compute palindromic decompositions for 
"04451881", we would recursively compute palindromic sequences for "4451991" 
(since "0" is the only prefix that is a palindrome). To compute palindromic decom¬ 
positions for "4451991", we would recursively compute palindromic sequences for 
"451991" (since "4" is a palindrome) and for "51991" (since "44' is a palindrome). 

public static List<List<String>> palindromePartitioning(String input) { 

List<List<String>> result = new ArrayList<>(); 

directedPalindromePartitioning(input, ©, new ArrayList<String>(), result); 
return result; 

} 

private static void directedPalindromePartitioning( 

String input, int offset, List<String> partialPartition, 

List<List<String>> result) { 
if (offset == input.length()) { 

result.add (new ArrayList<>(partialPartition)); 
return; 

} 

for (int i = offset +1; i <= input.length(); ++i) { 

String prefix = input.substring(offset, i); 
if (isPalindrome(prefix)) { 
partialPartition.add(prefix); 

directedPalindromePartitioning(input, i, partialPartition, result); 
partialPartition.remove(partialPartition.size() - 1); 

} 

} 

} 
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Figure 16.5: The five binary trees on three nodes. 


private static boolean isPalindrome(String prefix) { 

for (int i = ®, j = prefix.length() - 1; i < j; ++i, --j) { 
if (prefix.charAt(i) != prefix.charAt(j)) { 

return false; 

} 

} 

return true; 


The worst-case time complexity is still 0(n X 2"), e.g., if the input string consists of 
n repetitions of a single character. However, our program has much better best- 
case time complexity than the brute-force approach, e.g., when there are very few 
palindromic decompositions. 


16.8 Generate binary trees 

Write a program which returns all distinct binary trees with a specified number of 
nodes. For example, if the number of nodes is specified to be three, return the trees 
in Figure 16.5. 


Hint: Can two binary trees whose left subtrees differ in size be the same? 

Solution: A brute-force approach to generate all binary trees on n nodes would be to 
generate all binary trees on n - 1 or fewer nodes. Afterwards, form all binary trees 
with a root and a left child with n — 1 or fewer nodes, and a right child with n- 1 or 
fewer nodes. The resulting trees would all be distinct, since no two would have the 
same left and right child. However, some will have fewer than n- 1 nodes, and some 
will have more. 

The key to efficiency is to direct the search. If the left child has k nodes, we should 
only use right children with n - 1 — k nodes, to get binary trees with n nodes that 
have that left child. Specifically, we get all binary trees on n nodes by getting all left 
subtrees on i nodes, and right subtrees on n - 1 - i nodes, for i between 0 and n — 1. 

Looking carefully at Figure 16.5, you will see the first two trees correspond to the 
trees on three nodes which have a left subtree of size 0 and a right subtree of size 2. 
The third tree is the only tree on three nodes which has a left subtree of size 1 and a 
right subtree of size 1. The last two trees correspond to the trees on three nodes which 
have a left subtree of size 2 and a right subtree of size 0. The set of two trees on two 
nodes is itself computed recursively: there is a single binary tree on one node, and it 
may be on either side of the root. 
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public static List<BinaryTreeNode <Integer>> generateAUBinaryTrees ( 
int numNodes) { 

List<BinaryTreeNode<Integer>> result = new ArrayList<>(); 
if (numNodes == ®) { // Empty tree, add as an null. 
result. add(null) ; 

} 

for (int numLeftTreeNodes = ®; numLeftTreeNodes < numNodes; 
++numLeftTreeNodes) { 

int numRightTreeNodes = numNodes - 1 - numLeftTreeNodes; 

List <BinaryTree Node < Int eg er» left Subtrees 

= generateAUBinaryTrees (numLef tTreeNodes) ; 

List <BinaryTree Node < Int eg er» right Subtrees 

= generateAUBinaryTrees (numNodes - 1 - numLeftTreeNodes); 

// Generates all combinations of leftSubtrees and right Subtrees. 
for (BinaryTreeNodeclnteger> left : leftSubtrees) { 
for (BinaryTreeNode<Integer> right : rightSubtrees) { 
result.add(new BinaryTreeNode<>(®, left, right)); 

} 

} 

} 

return result; 


The number of calls C(n) to the recursive function satisfies the recurrence C(n) = 
E/Li C(n - i)C(i - 1). The quantity C(n) is called the nth Catalan number. It is known 
to be equal to . Comparing Solution 16.6 on Page 292 to this solution, you 
will see considerable similarity—the Catalan numbers appear in numerous types of 
combinatorial problems. 


16.9 Implement a Sudoku solver 

Implement a Sudoku solver. See Problem 6.16 on Page 85 for a definition of Sudoku. 
Hint: Apply the constraints to speed up a brute-force algorithm. 

Solution: A brute-force approach would be to try every possible assignment to empty 
entries, and then check if that assignment leads to a valid solution using Solution 6.16 
on Page 85. This is wasteful, since if setting a value early on leads to a constraint 
violation, there is no point in continuing. Therefore, we should apply the backtracking 
principle. 

Specifically, we traverse the 2D array entries one at a time. If the entry is empty, 
we try each value for the entry, and see if the updated 2D array is still valid; if it 
is, we recurse. If all the entries have been filled, the search is successful. The naive 
approach to testing validity is calling Solution 6.16 on Page 85. However, we can 
reduce runtime considerably by making use of the fact that we are adding a value to 
an array that already satisfies the constraints. This means that we need to check just 
the row, column, and subgrid of the added entry. 

For example, suppose we begin with the lower-left entry for the configuration in 
Figure 6.2(a) on Page 86. Adding a 1 to entry does not violate any row, column, or 


296 



subgrid constraint, so we move on to the next entry in that row. We cannot put a 1, 
since that would now violate a row constraint; however, a 2 is acceptable. 

private static final int EMPTY_ENTRY = ®; 

public static boolean solveSudoku(List<List<Integer>> partialAssignment) { 
return solvePartialSudoku(®, ®, partialAssignment); 

} 

private static boolean solvePartialSudoku( 

int i, int j, List<List<Integer>> partialAssignment) { 
if (i == partialAssignment.size()) { 
i = ®; // Starts a new row. 

if (++j == partialAssignment.get(i).size()) { 

return true; // Entire matrix has been filled without conflict. 

} 

} 

// Skips nonempty entries. 

if (partialAssignment.get(i).get(j) != EMPTY.ENTRY) { 
return solvePartialSudoku(i +1, j, partialAssignment); 

} 

for (int val = 1; val <= partialAssignment.size(); ++val) { 

// It’s substantially quicker to check if entry val conflicts 

// with any of the constraints if we add it at (i,j) before 

// adding it, rather than adding it and then checking all constraints. 

// The reason is that we are starting with a valid configuration, 

// and the only entry which can cause a problem is entryval at (i,j). 
if (validToAddVal(partialAssignment, i, j, val)) { 
partialAssignment.get(i).set(j, val); 

if (solvePartialSudoku(i + 1, j, partialAssignment)) { 
return true; 

} 

} 

} 

partialAssignment.get(i).set(j f EMPTY_ENTRY); // Undo assignment. 

return false; 


private static boolean validToAddVal(List<Listdnteger» partialAssignment, 

int i, int j, int val) { 

// Check row constraints. 

for (Listdnteger> element : partialAssignment) { 
if (val == element.get(j)) { 

return false; 

} 

} 

// Check column constraints. 

for (int k = ®; k < partialAssignment.size(); ++k) { 
if (val == partialAssignment.get(i).get (k)) { 

return false; 

} 

} 
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// Check region constraints. 

int regionSize = (int)Math .sqrt(partialAssignment.size()); 
int I = i / regionSize, J = j / regionSize; 
for (int a = ®; a < regionSize; ++a) { 
for (int b = ®; b < regionSize; ++b) { 
if (val == partialAssignment 

.get(regionSize * I + a).get(regionSize * J + b)) { 

return false; 

} 

} 

} 

return true; 


Because the program is specialized to 9 X 9 grids, it does not make sense to speak of its 
time complexity, since there is no notion of scaling with a size parameter. However, 
since the problem of solving Sudoku generalized t o n X n grids is NP-complete, it 
should not be difficult to prove that the generalization of this algorithm tonXn grids 
has exponential time complexity. 


16.10 Compute a Gray code 

An n-bit Gray code is a permutation of {0,1,2,... ,2” - 1} such that the binary rep¬ 
resentations of successive integers in the sequence differ in only one place. (This 
is with wraparound, i.e., the last and first elements must also differ in only one 
place.) For example, both <(000) 2 ,(100)2,(101)2,(111)2,(110)2,(010)2,(011) 2 ,(001) 2 ) = 
(0,4,5,7,6,2,3,1) and (0,1,3,2,6,7,5,4) are Gray codes for n = 3. 

Write a program which takes n as input and returns an n-bit Gray code. 

Hint: Write out Gray codes for n = 2,3,4. 

Solution: A brute-force approach would be to enumerate sequences of length 2" 
whose entries are n bit integers. We would test if the sequence is a Gray code, and stop 
as soon as we find one. The complexity is astronomical—there are 2" x2 ” sequences. 
We can improve complexity by enumerating permutations of 0,1,2,..., 2” - 1, since 
we want distinct entries, but this is still a very high complexity. 

We can do much better by directing the enumeration. Specifically, we build the 
sequence incrementally, adding a value only if it is distinct from all values currently 
in the sequence, and differs in exactly one place with the previous value. (For the 
last value, we have to check that it differs in one place from the first value.) This is 
the approach shown below. For n- 4, we begin with (0000) 2 . Next we try changing 
bits in (0000) 2 , one-at-a-time to get a value not currently present in the sequence, 
which yields (0001) 2 , which we append to the sequence. Changing bits in (0001) 2 , 
one-at-a-time, we get (0000) 2 (which is already present), and then (0011) 2 , which is 
not, so we append it to the sequence, which is now ((0000) 2 , (0001) 2 , (0011) 2 ). The 
next few values are (0010) 2 , (0011) 2 , (0111) 2 . 

public static List<Integer> grayCode(int numBits) { 

List<Integer> result = new ArrayList<>(Arrays.asList(®)); 

directedGrayCode(numBits, new HashSet<Integer>(Arrays.asList(®)), result); 
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return result; 


private static boolean directedGrayCode (int numBits, Setdnteger> history, 

Listdnteger> result) { 

if (result.size() == (1 « numBits)) { 

return differsByOneBit(result.get(®), result.get(result.size() - 1)); 

} 

for (int i = ®; i < numBits; ++i) { 

int previousCode = result.get(result.size() - 1); 
int candidateNextCode = previousCode A (1 « i); 
if (!history.contains(candidateNextCode)) { 
history.add(candidateNextCode); 
result.add(candidateNextCode); 

if (directedGrayCode(numBits, history, result)) { 

return true; 

} 

history.remove(candidateNextCode); 
result.remove(result.size() - 1); 

} 

} 

return false; 

} 

private static boolean differsByOneBit (int x, int y) { 
int bitDifference = x A y; 

return bitDifference != ® && (bitDifference & (bitDifference - 1)) == ®; 


Now we present a more analytical solution. The inspiration comes from small 
case analysis. The sequence ((00) 2 ,(01) 2 , (11) 2 ,(10) 2 ) is a 2-bit Gray code. To get 
to n = 3, we cannot just prepend 0 to each elements of ((00) 2 ,(01) 2/ (11) 2 ,(10) 2 ), 
1 to ((00) 2 ,(01) 2 , (11) 2 , (10) 2 ) and concatenate the two sequences—that leads to the 
Gray code property being violated from (010) 2 to (100) 2 . However, it is preserved 
everywhere else. 

Since Gray codes differ in one place on wrapping around, prepending 1 
to the reverse of ((00) 2 , (01) 2 ,(11) 2 , (10) 2 ) solves the problem when transition¬ 
ing from a leading 0 to a leading 1. For n - 3 this leads to the sequence 
((000) 2 , (001) 2 , (011) 2 , (010) 2 , (110) 2 , (111)2, (101)2, (100) 2 ). The general solution uses re¬ 
cursion in conjunction with this reversing, and is presented below. 

public static List<Integer> grayCode(int numBits) { 
if (numBits == ®) { 

return new ArrayList<>(Arrays.asList(®)); 

} 

// These implicitly begin with ® at bit-index (numBits - 1 ). 

Listdnteger> grayCodeNumBitsMinus1 = grayCode(numBits - 1); 

// Now, add a 1 at bit-index (numBits - 1) to all entries in 

// grayCodeNumBitsMinus1. 

int leadingBitOne = 1 « (numBits - 1); 

// Process in reverse order to achieve reflection of grayCodeNumBitsMinus1. 
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for (int i = grayCodeNumBitsMinus1.size() - 1; i >= Q; --i) { 

grayCodeNumBitsMinus1.add(leadingBitOne | grayCodeNumBitsMinus1.get(i)); 

} 

return grayCodeNumBitsMinus1; 


Assuming we operate on integers that fit within the size of the integer word, the time 
complexity T(n) satisfies T(n) = T(n - 1) + <9(2" _1 ). the time complexity is <9(2”). 


16.11 Compute the diameter of a tree 

Packets in Ethernet local area networks (LANs) are routed according to the unique 
path in a tree whose leaves correspond to clients, internal nodes to switches, and 
edges to physical connection. In this problem, we want to design an algorithm for 
finding the "worst-case" route, i.e., the two clients that are furthest apart. 



Figure 16.6: The diameter for the above tree is 31. The corresponding path is (A,B,C,D,E), which is 
depicted by the dashed edges. 


The diameter of a tree is defined to be the length of a longest path in the tree. 
Figure 16.6 illustrates the diameter concept. Design an efficient algorithm to compute 
the diameter of a tree. 

Hint: The longest path may or may not pass through the root. 

Solution: We can compute the diameter with a brute-force approach. Specifically, we 
can run BFS, described on Page 353, from each node, recording the maximum value of 
the shortest path distances computed. BFS in a graph with n vertices and m edges has 
time complexity 0(m+n). Therefore, the brute-force algorithm has 0(n(m+n)) = 0(n 2 ) 
time complexity since the number of edges in a tree is one less than the number of 
vertices. 

The above approach is inefficient, since it repeatedly processes the same vertices 
and edges. We can achieve better time complexity by using divide-and-conquer. 
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Consider a longest path in the tree. Either it passes through the root or it does not 
pass through the root. 

• If the longest path does not pass through the root, it must be entirely within one 
of the subtrees. Therefore, in this case, the longest path length in the tree is the 
maximum of the diameters of the subtrees. 

• If the longest path does pass through the root, it must be between a pair of 
nodes in distinct subtrees that are furthest from the root. The distance from the 
root to the node in the zth subtree T, that is furthest from it is fi = hi + where 
hi is the height of T, and /, is the length of the edge from the root to the root of 
TV 

Since one of the two cases must hold, the longest length path is the larger of the 
maximum of the subtree diameters and the sum of the two largest fis. 

The base cases correspond to a tree that has no children, in which case the length 
of the longest path is 0. 

For the tree in Figure 16.6 on the facing page, the subtree diameters, computed 
recursively, are 13, 0, and 15 (from left to right). The heights are 10, 0, and 10 (from 
left to right). The distance from the root to the furthest node of the first subtree is 
7 + 10 = 17, the distance from the root to the furthest node of the second subtree is 
14 + 0 = 14, and the distance from the root to the furthest node of the third subtree is 
3 + 10 = 13. The largest diameter in a subtree is 15, which is less than the sum of the 
two greatest distances (17 + 14 = 31), so the diameter of the tree is 31. 


public static class TreeNode { List<Edge> edges = new ArrayList<>(); } 

private static class Edge { 
public TreeNode root; 
public Double length; 

public Edge(TreeNode root, Double length) { 
this .root = root; 
this. length = length; 

} 

} 

private static class HeightAndDiameter { 
public Double height; 
public Double diameter; 

public HeightAndDiameter(Double height, Double diameter) { 
this. height = height; 
this .diameter = diameter; 

} 

} 

public static double computeDiameter(TreeNode T) { 

return T != null ? computeHeightAndDiameter(T).diameter : ®.®; 

} 

private static HeightAndDiameter computeHeightAndDiameter(TreeNode r) { 
double diameter = Double.MIN_VALUE; 

doublet] heights = {®.®, ®.®}; // Stores the max two heights, 
for (Edge e : r.edges) { 
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HeightAndDiameter heightDiameter = computeHeightAndDiameter(e.root); 
if (heightDiameter.height + e.length > heights[®]) { 
heights[1] = heights [®] ; 

heights[®] = heightDiameter.height + e.length; 

} else if (heightDiameter.height + e.length > heights[l]) { 
heights[1] = heightDiameter.height + e.length; 


} 


diameter = Math.max(diameter, heightDiameter.diameter); 

} 

return new HeightAndDiameter(heights[®], 

Math.max(diameter f heights[®] 


heights[1])) ; 


Since the time spent at each node is proportional to the number of its children, the 
time complexity is proportional to the size of the tree, i.e., 0(n). 

Variant: Consider a computer network organized as a rooted tree. A node can send a 
message to only one child at a time, and it takes one second for the child to receive the 
message. The root periodically receives a message from an external source. It needs 
to send this message to all the nodes in the tree. The root has complete knowledge of 
how the network is organized. Design an algorithm that computes the sequence of 
transfers that minimizes the time taken to transfer a message from the root to all the 
nodes in the tree. 
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Chapter 


Dynamic Programming 

The important fact to observe is that we have attempted to solve 
a maximization problem involving a particular value of x and 
a particular value of N by first solving the general problem 
involving an arbitrary value of x and an arbitrary value ofN. 

— "Dynamic Programming," 
R. E. Bellman, 1957 


DP is a general technique for solving optimization, search, and counting problems 
that can be decomposed into subproblems. You should consider using DP whenever 
you have to make choices to arrive at the solution, specifically, when the solution 
relates to subproblems. 

Like divide-and-conquer, DP solves the problem by combining the solutions of 
multiple smaller problems, but what makes DP different is that the same subproblem 
may reoccur. Therefore, a key to making DP efficient is caching the results of inter¬ 
mediate computations. Problems whose solutions use DP are a popular choice for 
hard interview questions. 

To illustrate the idea underlying DP, consider the problem of computing Fi¬ 
bonacci numbers. The first two Fibonacci numbers are 0 and 1. Successive num¬ 
bers are the sums of the two previous numbers. The first few Fibonacci numbers are 
0,1,1,2,3,5,8,13,21, — The Fibonacci numbers arise in many diverse applications— 
biology, data structure analysis, and parallel computing are some examples. 

Mathematically, the nth Fibonacci number F(n) is given by the equation F(n) = 
F(n - 1) + F(n - 2), with F(0) = 0 and F(l) = 1. A function to compute F(n) that 
recursively invokes itself has a time complexity that is exponential in n. This is 
because the recursive function computes some F(i) s repeatedly. Figure 17.1 on the 
next page graphically illustrates how the function is repeatedly called with the same 
arguments. 

Caching intermediate results makes the time complexity for computing the nth 
Fibonacci number linear in n, albeit at the expense of 0(n) storage. 

private static Mapdnteger, Integer> cache = new HashMap<>() ; 

public static int fibonacci (int n) { 

if (n <= 1) { 
return n; 

} else if (!cache.containsKey(n)) { 

cache.put(n, fibonacci(n - 2) + fibonacci(n - 1)); 

} 

return cache.get(n); 
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Figure 17.1: Tree of recursive calls when naively computing the 5th Fibonacci number, F(5). Each node 
is a call: F(x) indicates a call with argument x, and the italicized numbers on the right are the sequence 
in which calls take place. The children of a node are the subcalls made by that call. Note how there are 
2 calls to F( 3), and 3 calls to each of F( 2), F( 1), and F(0). 


} 


Minimizing cache space is a recurring theme in DP. Now we show a program that 
computes F(n) in 0(n) time and (9(1) space. Conceptually, in contrast to the above 
program, this program iteratively fills in the cache in a bottom-up fashion, which 
allows it to reuse cache storage to reduce the space complexity of the cache. 

public static int fibonacci (int n) { 
if (n <= 1) { 
return n; 

} 

int fMinus2 = Q; 
int fMinus 1 = 1; 
for (int i = 2; i <= n; ++i) { 
int f = fMinus2 + fMinusl; 
fMinus2 = fMinusl; 
fMinusl = f; 

} 

return fMinusl; 


The key to solving a DP problem efficiently is finding a way to break the problem 
into subproblems such that 

• the original problem can be solved relatively easily once solutions to the sub¬ 
problems are available, and 

• these subproblem solutions are cached. 

Usually, but not always, the subproblems are easy to identify. 

Here is a more sophisticated application of DP. Consider the following problem: 
find the maximum sum over all subarrays of a given array of integer. As a concrete 
example, the maximum subarray for the array in Figure 17.2 on the facing page starts 
at index 0 and ends at index 3. 

The brute-force algorithm, which computes each subarray sum, has <9(n 3 ) time 
complexity—there are w( ” 2 +1) subarrays, and each subarray sum takes 0(n) time to 
compute. The brute-force algorithm can be improved to 0(n 2 ), at the cost of 0(n) 
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Figure 17.2: An array with a maximum subarray sum of 1479. 


additional storage, by first computing S[k] = £ A[0 : k] for all k. The sum for A[i : j] 
is then S[j] - S[i - 1], where S[—1] is taken to be 0. 

Here is a natural divide-and-conquer algorithm. Take m = |_fj to be the middle 
index of A. Solve the problem for the subarrays L = A[0 : m] and R = A[m + 1 : n - 1]. 
In addition to the answers for each, we also return the maximum subarray sum / for a 
subarray ending at the last entry in L, and the maximum subarray sum r for a subarray 
starting at the first entry of R. The maximum subarray sum for A is the maximum of 
/ + r, the answer for L, and the answer for R. The time complexity analysis is similar to 
that for quicksort, and the time complexity is 0(n log n). Because of off-by-one errors, 
it takes some time to get the program just right. 

Now we will solve this problem by using DP. A natural thought is to assume we 
have the solution for the subarray A[0 : n — 2]. However, even if we knew the largest 
sum subarray for subarray A[0 : n - 2], it does not help us solve the problem for 
A[0 : n - 1]. Instead we need to know the subarray amongst all subarrays A[0 : /], 
i < n - 1, with the smallest subarray sum. The desired value is S[n - 1] minus this 
subarray's sum. We compute this value by iterating through the array. For each index 
j, the maximum subarray sum ending at j is equal to S[j] - min fc <y S[k]. During the 
iteration, we track the minimum S[k] we have seen so far and compute the maximum 
subarray sum for each index. The time spent per index is constant, leading to an 0(n) 
time and (9(1) space solution. It is legal for all array entries to be negative, or the 
array to be empty. The algorithm handles these input cases correctly 

public static int findMaximumSubarray(List<Integer> A) { 
int minSum = ®, sum = ©, maxSum = ©; 
for (int i = ®; i < A.sizeO; ++i) { 
sum += A . get (i) ; 
if (sum < minSum) { 
minSum = sum; 

} 

if (sum - minSum > maxSum) { 
maxSum = sum - minSum; 

} 

} 

return maxSum; 

} 


Here are some common mistakes made when applying DP. 

• A common mistake in solving DP problems is trying to think of the recursive 
case by splitting the problem into two equal halves, a la quicksort, i.e., solve 
the subproblems for subarrays A[0 : |_f J] and A[|_fJ + 1 : n] and combine the 
results. However, in most cases, these two subproblems are not sufficient to 
solve the original problem. 
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• Make sure that combining solutions to the subproblems does yield a solution 
to the original problem. For example, if the longest path without repeated 
cities from City 1 to City 2 passes through City 3, then the subpaths from City 1 
to City 3 and City 3 to City 2 may not be individually longest paths without 
repeated cities. 


Dynamic programming boot camp 

The programs presented in the introduction for computing the Fibonacci numbers 
and the maximum subarray sum are good illustrations of DP. 


Consider using DP whenever you have to make choices to arrive at the solution, 
specifically, when the solution relates to subproblems. 

In addition to optimization problems, DP is also applicable to counting and 
decision problems —any problem where you can express a solution recursively 
in terms of the same computation on smaller instances. 

Although conceptually DP involves recursion, often for efficiency the cache is 
built "bottom-up", i.e., iteratively. [Problem 17.3]. 

To save space, cache space may be recycled once it is known that a set of entries 
will not be looked up again. [Problems 17.1 and 17.2] 

Sometimes, recursion may out-perform a bottom-up DP solution, e.g., when 
the solution is found early or subproblems can be pruned through bounding. 
[Problem 17.5] 


17.1 Count the number of score combinations 

In an American football game, a play can lead to 2 points (safety), 3 points (field goal), 
or 7 points (touchdown, assuming the extra point). Many different combinations of 
2, 3, and 7 point plays can make up a final score. For example, four combinations of 
plays yield a score of 12: 

• 6 safeties (2 x 6 = 12), 

• 3 safeties and 2 field goals (2 X 3 + 3 X 2 = 12), 

• 1 safety, 1 field goal and 1 touchdown (2xl + 3xl + 7xl = 12), and 

• 4 field goals (3 x 4 = 12). 

Write a program that takes a final score and scores for individual plays, and returns 
the number of combinations of plays that result in the final score. 

Hint: Count the number of combinations in which there are 0 zoq plays, then 1 Wq plays, etc. 

Solution: We can gain some intuition by considering small scores. For example, a 9 
point score can be achieved in the following ways: 

• scoring 7 points, followed by a 2 point play, 

• scoring 6 points, followed by a 3 point play, and 

• scoring 2 points, followed by a 7 point play. 
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Generalizing, an s point score can be achieved by an s - 2 point score, followed by a 

2 point play, an s - 3 point score, followed by a 3 point play, or an s - 7 point score, 
followed by a 7 point play. This gives us a mechanism for recursively enumerating all 
possible scoring sequences which lead to a given score. Note that different sequences 
may lead to the same score combination, e.g., a 2 point play followed by a 7 point 
play and a 7 point play followed by a 2 point play both lead to a final score of 9. A 
brute-force approach might be to enumerate these sequences, and count the distinct 
combinations within these sequences, e.g., by sorting each sequence and inserting 
into a hash table. 

The time complexity is very high, since there may be a very large number of scoring 
sequences. Since all we care about are the combinations, a better approach is to focus 
on the number of combinations for each possible number of plays of a single type. 

For example, if the final score is 12 and we are only allowed 2 point plays, there 
is exactly one way to get 12. Now suppose we are allowed both 2 and 3 point plays. 
Since all we care about are combinations, assume the 2 point plays come before the 3 
point plays. We could have zero 2 point plays, for which there is one combination of 

3 point plays, one 2 point play, for which there no combination of 3 point plays (since 
3 does not evenly divide 12-2, two 2 point plays, for which there no combination 
of 3 point plays (since 3 does not evenly 12 - 2 X 2), three 2 point plays, for which 
there is one combination of 3 point plays (since 3 evenly divides 12 - 2 X 3), etc. To 
count combinations when we add 7 point plays to the mix, we add the number of 
combinations of 2 and 3 that lead to 12 and to 5—these are the only scores from which 
7 point plays can lead to 12. 

Naively implemented, for the general case, i.e., individual play scores are 
W[0], W[l], ", W[w - 1], and s the final score, the approach outlined above has expo¬ 

nential complexity because it repeatedly solves the same problems. We can use DP to 
reduce its complexity. Let the 2D array A[i][j] store the number of score combinations 
that result in a total of j, using individual plays of scores W[0], W[l],..., W[i - 1]. For 
example, A[l][12] is the number of ways in which we can achieve a total of 12 points, 
using 2 and/or 3 point plays. Then A[i +1 ][j] is simply A[i][j] (no W[i + 1] point plays 
used to get to j), plus A[i][j- W[i + 1]] (one W[i +1] point play), plus A[i][j- 2W[i + 1]] 
(two W[i + 1] point plays), etc. 

The algorithm directly based on the above discussion consists of three nested loops. 
The first loop is over the total range of scores, and the second loop is over scores for 
individual plays. For a given i and j, the third loop iterates over at most j/W[i\ + 1 
values. (For example, if j = 10, i = 1, and W[i] = 3, then the third loop examines 
A[0][10], A[0][7], A[0][4], A[0][1].) Therefore number of iterations for the third loop is 
bounded by s. Hence, the overall time complexity is 0(sns) = 0(s 2 n) (first loop is to 
n, second is to s, third is bounded by s). 

Looking more carefully at the computation for the row A[i +1], it becomes apparent 
that it is not as efficient as it could be. As an example, suppose we are working with 
2 and 3 point plays. Suppose we are done with 2 point plays. Let A[0] be the row 
holding the result for just 2 point plays, i.e., A[0][j] is the number of combinations 
of 2 point plays that result in a final score of j. The number of score combinations 
to get a final score of 12 when we include 3 point plays in addition to 2 point plays 
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is A[0][0] + A[0][3] + A[0][6] + A[0][9] + A[0][12]. The number of score combinations 
to get a final score of 15 when we include 3 point plays in addition to 2 point plays 
is A[0][0] + A[0][3] + A[0][6] + A[0][9] + A[0][12] + A[0][15]. Clearly this repeats 
computation—A[0][0] + A[0][3] + A[0][6] + A[0][9] + A[0][12] was computed when 
considering the final score of 12. 

Note that A[l][15] = A[0][15] + A[l][12]. Therefore a better way to fill in A[l] is 
as follows: A[1][0] = A[0][0], A[l][l] = A[0][1],A[1][2] = A[0][2],A[1][3] = A[0][3] + 
A[1][0],A[1][4] = A[0][4]+A[1][1],A[1][5] = A[0][5]+A[l][2],.... Observe that A[l][i] 
takes 0(1) time to compute—it's just A[0][i]+A[l][i—3]. See Figure 17.3 for an example 
table. 


0123456789 10 11 12 

2 
2,3 
2,3,7 


Figure 17.3: DP table for 2,3,7 point plays (rows) and final scores from 0 to 12 (columns). As an 
example, for 2,3,7 point plays and a total of 9, the entry is the sum of the entry for 2,3 point plays and 
a total of 9 (no 7 point plays) and the entry for 2,3,7 point plays and a total of 2 (one additional 7 point 
play). 
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The code below implements the generalization of this approach. 


public static int numCombinationsForFinalScore( 

int finalScore, List<Integer> individualPlayScores) { 
int[][] numCombinationsForScore 

= new int [individualPlayScores.size()][finalScore + 1]; 
for (int i = Q; i < individualPlayScores.size(); ++i) { 
numCombinationsForScore [i] [©] = 1; // One way to reach (9. 
for (int j = 1 ; j <= finalScore; ++ j ) { 
int withoutThisPlay 

= i - 1 >= ® ? numCombinationsForScore[i - l][j] : ®; 

int withThisPlay 

= j >= individualPlayScores.get(i) 

? numCombinationsForScore[i][j - individualPlayScores.get(i)] 

: ®; 

numCombinationsForScore[i][j] = withoutThisPlay + withThisPlay; 

} 

} 

return numCombinationsForScore[individualPlayScores.size() - 1][finalScore]; 

} 


The time complexity is 0(sn) (two loops, one to s, the other to n) and the space 
complexity is 0(sn) (the size of the 2D array). 

Variant: Solve the same problem using 0(s) space. 

Variant: Write a program that takes a final score and scores for individual plays, and 
returns the number of sequences of plays that result in the final score. For example, 
18 sequences of plays yield a score of 12. Some examples are (2,2,2,3,3), (2,3,2,2,3), 
(2,3,7), (7,3,2). 
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Variant: Suppose the final score is given in the form (s, s'), i.e.. Team 1 scored s points 
and Team 2 scored s' points. How would you compute the number of distinct scoring 
sequences which result in this score? For example, if the final score is (6,3) then 
Team 1 scores 3, Team 2 scores 3, Team 1 scores 3 is a scoring sequence which results 
in this score. 

Variant: Suppose the final score is (s,s'). How would you compute the maximum 
number of times the team that lead could have changed? For example, if s = 10 and 
s' = 6, the lead could have changed 4 times: Team 1 scores 2, then Team 2 scores 3 
(lead change), then Team 1 scores 2 (lead change), then Team 2 scores 3 (lead change), 
then Team 1 scores 3 (lead change) followed by 3. 

Variant: You are climbing stairs. You can advance 1 to k steps at a time. Your 
destination is exactly n steps up. Write a program which takes as inputs n and k and 
returns the number of ways in which you can get to your destination. For example, 
if n = 4 and k = 2, there are five ways in which to get to the destination: 

• four single stair advances, 

• two single stair advances followed by a double stair advance, 

• a single stair advance followed by a double stair advance followed by a single 
stair advance, 

• a double stair advance followed by two single stairs advances, and 

• two double stair advances. 

17.2 Compute the Levenshtein distance 

Spell checkers make suggestions for misspelled words. Given a misspelled string, a 
spell checker should return words in the dictionary which are close to the misspelled 
string. 

In 1965, Vladimir Levenshtein defined the distance between two words as the 
minimum number of "edits" it would take to transform the misspelled word into a 
correct word, where a single edit is the insertion, deletion, or substitution of a single 
character. For example, the Levenshtein distance between "Saturday" and "Sundays" 
is 4—delete the first 'a' and 't', substitute 'r' by 'n' and insert the trailing 's'. 

Write a program that takes two strings and computes the minimum number of edits 
needed to transform the first string into the second string. 

Hint: Consider the same problem for prefixes of the two strings. 

Solution: A brute-force approach would be to enumerate all strings that are distance 
1,2,3,... from the first string, stopping when we reach the second string. The number 
of strings may grow enormously, e.g., if the first string is n Os and the second is n Is 
we will visit all of the 2” possible bit strings from the first string before we reach the 
second string. 

A better approach is to "prune" the search. For example, if the last character of the 
first string equals the last character of the second string, we can ignore this character. 
If they are different, we can focus on the initial portion of each string and perform a 
final edit step. (As we will soon see, this final edit step may be an insertion, deletion, 
or substitution.) 
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Let a and b be the length of strings A and B, respectively. LetE(A[0 : a-l],B[0 : b— 1]) 
be the Levenshtein distance between the strings A and B. (Note that A [0 : a - 1] is 
just A, but we prefer to write A using the subarray notation for exposition; we do the 
same for B.) 

We now make some observations: 

• If the last character of A equals the last character of B, then E(A[0 : a - 1], B[0 : 
b - 1]) = E(A[ 0 : a — 2],B[0 : b - 2]). 

• If the last character of A is not equal to the last character of B then 


E(A[ 0 : a - 1],B[0 :b — !]) = ! + min 


' E{A [0 
E(A[ 0 
k E(A[0 


a-2],B[0:b-2]), ' 
a-l],B[0:b-2]), 
a ~ 2]/B[0 :b — l]) , 


The three terms correspond to transforming A to B by the following three ways: 

- Transforming A[0 : a - 1] to B[0 : b - 1] by transforming A [0 : a - 2] to 
B[0 : b - 2] and then substituting A's last character with B's last character. 

- Transforming A[0 : a - 1] to B[0 : b - 1] by transforming A [0 : a - 1] to 
B[0 : b - 2] and then adding B's last character at the end. 

- Transforming A[0 : a - 1] to B[0 : b - 1] by transforming A [0 : a - 2] to 
B[0 : b - 1] and then deleting A's last character. 

These observations are quite intuitive. Their rigorous proof, which we do not 
give, is based on reordering steps in an arbitrary optimum solution for the original 
problem. 

DP is a great way to solve this recurrence relation: cache intermediate results on 
the way to computing E(A[0 : a - 1],B[0 : b - 1]). 

We illustrate the approach in Figure 17.4 on the facing page. This shows the E 
values for "Carthorse" and "Orchestra". Uppercase and lowercase characters are 
treated as being different. The Levenshtein distance for these two strings is 8. 


public static int levenshteinDistance(String A, String B) { 

int[][] distanceBetweenPrefixes = new int [A.length()][B.length()]; 
for (int[] row : distanceBetweenPrefixes) { 

Arrays.fill(row, -1); 

} 

return computeDistanceBetweenPref ixes (A, A.lengthO - 1, B, B.lengthO 

distanceBetweenPrefixes); 


} 


1 , 


private static int computeDistanceBetweenPrefixes( 

String A, int A_idx, String B, int B_idx, 
int[][] distanceBetweenPrefixes) { 
if (A_idx < ®) { 

// A is empty so add all of B’s characters . 
return B_idx + 1; 

} else if (B_idx < ®) { 

// B is empty so delete all of A’s characters. 
return A_idx + 1; 

} 

if (distanceBetweenPrefixes[A_idx][B_idx] == -1) { 
if (A.charAt(A_idx) == B.charAt(B_idx)) { 

distanceBetweenPrefixes[A_idx][B_idx] = computeDistanceBetweenPrefixes( 
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1, distanceBetweenPrefixes); 


A, A_idx - 1, B, B_idx - 
} else { 

int substituteLast = computeDistanceBetweenPrefixes( 

A, A_idx - 1, B, B_idx - 1, distanceBetweenPrefixes); 
int addLast = computeDistanceBetweenPrefixes(A, A_idx, B, B_idx - 1, 

distanceBetweenPrefixes); 
int deleteLast = computeDistanceBetweenPrefixes( 

A, A_idx - 1, B, B_idx, distanceBetweenPrefixes); 
distanceBetweenPrefixes[A_idx][B_idx] 

= 1 + Math.min(substituteLast, Math.min(addLast, deleteLast)); 

} 

} 

return distanceBetweenPrefixes[A_idx][B_idx]; 

} 


The value E(A[0 : a-l],B[0 : b-1]) takes time (9(1) to compute once E(A[0 : k],B [0 : /]) 
is known for all k < a and l < b. This implies 0(ab) time complexity for the algorithm. 
Our implementation uses 0(ab) space. 



Figure 17.4: The E table for “Carthorse” and “Orchestra”. 


Variant: Compute the Levenshtein distance using <9(min(fl, b)) space and 0(ab) time. 

Variant: Given A and B as above, compute a longest sequence of characters that is a 
subsequence of A and of B. For example, the longest subsequence which is present 
in both strings in Figure 17.4 is (r, h, s). 

Variant: Given a string A, compute the minimum number of characters you need to 
delete from A to make the resulting string a palindrome. 

Variant: Given a string A and a regular expression r, what is the string in the language 
of the regular expression r that is closest to A? The distance between strings is the 
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Levenshtein distance specified above. 

Variant: Define a string t to be an interleaving of strings s\ and S 2 if there is a way to 
interleave the characters of si and s 2 , keeping the left-to-right order of each, to obtain 
t. For example, if si is "gtaa" and s 2 is "ate", then "gattaca" and "gtataac" can be 
formed by interleaving s\ and s 2 but "gatacta" cannot. Design an algorithm that takes 
as input strings s lr s 2 and t, and determines if t is an interleaving of si and s 2 . 

17.3 Count the number of ways to traverse a 2D array 

In this problem you are to count the number of ways of starting at the top-left corner 
of a 2D array and getting to the bottom-right comer. All moves must either go right 
or down. For example, we show three ways in a 5 X 5 2D array in Figure 17.5. (As we 
will see, there are a total of 70 possible ways for this example.) 





Figure 17.5: Paths through a 2D array. 


Write a program that counts how many ways you can go from the top-left to the 
bottom-right in a 2D array. 

Hint: If i > 0 and j > 0, you can get to (i, j) from (i - 1, j ) or (j - 1, i). 

Solution: A brute-force approach is to enumerate all possible paths. This can be done 
using recursion. However, there is a combinatorial explosion in the number of paths, 
which leads to huge time complexity. 

The problem statement asks for the number of paths, so we focus our attention 
on that. A key observation is that because paths must advance down or right, the 
number of ways to get to the bottom-right entry is the number of ways to get to the 
entry immediately above it, plus the number of ways to get to the entry immediately 
to its left. Let's treat the origin (0,0) as the top-left entry. Generalizing, the number 
of ways to get to (/, j) is the number of ways to get to (i - 1, j) plus the number of 
ways to get to (i,j - 1). (If / = 0 or j = 0, there is only one way to get to ( i,j ) from 
the origin.) This is the basis for a recursive algorithm to count the number of paths. 
Implemented naively, the algorithm has exponential time complexity—it repeatedly 
recurses on the same locations. The solution is to cache results. For example, the 
number of ways to get to (z, j) for the configuration in Figure 17.5 is cached in a matrix 
as shown in Figure 17.6 on the facing page. 

public static int numberOfWays (int n, int m) { 

return computeNumberOfWaysToXY(n - 1, m - 1, new int[n][m]); 

} 
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private static int computeNumberOfWaysToXY(int x, int y, 

int[][] numberOfWays) { 

if (x == 8 I I y == 8) { 

return 1; 

} 

if (numberOfWays[x][y] == ®) { 
int waysTop 

= x == ® ? ® : computeNumberOfWaysToXY(x - 1, y, numberOfWays); 
int waysLeft 

= x == ® ? ® : computeNumberOfWaysToXY(x, y - 1, numberOfWays); 
numberOfWays[x][y] = waysTop + waysLeft; 

} 

return numberOfWays[x][y]; 


The time complexity is 0(nm), and the space complexity is 0(nm), where n is the 
number of rows and m is the number of columns. 


1 1 

1 

1 

1 

1 2 

3 

4 

5 

1 3 

6 

10 

15 

1 4 

10 

20 

35 

1 5 

15 

35 

70 


Figure 17.6: The number of ways to get from (0,0) to (i, j) for 0 < i, j < 4. 

A more analytical way of solving this problem is to use the fact that each path from 
(0,0) to (n - 1, m - 1) is a sequence of m - 1 horizontal steps and n — 1 vertical steps. 
There are (" + „?' 2 ) = ("^' 2 ) = such paths. 

Variant: Solve the same problem using 0( min(rc, m)) space. 

Variant: Solve the same problem in the presence of obstacles, specified by a Boolean 
2D array, where the presence of a true value represents an obstacle. 

Variant: A fisherman is in a rectangular sea. The value of the fish at point (/, j) in the 
sea is specified by an n X m 2D array A. Write a program that computes the maximum 
value of fish a fisherman can catch on a path from the upper leftmost point to the 
lower rightmost point. The fisherman can only move down or right, as illustrated in 
Figure 17.7 on the next page. 

Variant: Solve the same problem when the fisherman can begin and end at any point. 
He must still move down or right. (Note that the value at (i, j) may be negative.) 

Variant: A decimal number is a sequence of digits, i.e., a sequence over {0,1,2,..., 9}. 
The sequence has to be of length 1 or more, and the first element in the sequence 
cannot be 0. Call a decimal number D monotone if D[i] < D[i + 1],0 < i < |D|. Write 
a program which takes as input a positive integer k and computes the number of 
decimal numbers of length k that are monotone. 
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Figure 17.7: Alternate paths for a fisherman. Different types of fish have different values, which are 
known to the fisherman. 


Variant: Call a decimal number D, as defined above, strictly monotone if D[i] < D[i + 
1],0<I <\D\. Write a program which takes as input a positive integer k and computes 
the number of decimal numbers of length k that are strictly monotone. 


17.4 Compute the binomial coefficients 

The symbol (”) is the short form for the expression • It is the number of 

ways to choose a /c-element subset from an n-element set. It is not obvious that the 
expression defining (”) always yields an integer. Furthermore, direct computation of 
Q from this expression quickly results in the numerator or denominator overflowing 
if integer types are used, even if the final result fits in a 32-bit integer. If floats are 
used, the expression may not yield a 32-bit integer. 

Design an efficient algorithm for computing (”) which has theproperty that it never 
overflows if the final result fits in the integer word size. 

Hint: Write an equation. 

Solution: A brute-force approach would be to compute n(n - 1) • • • (n - k + 1), then 
k(k - 1) • • • (3)(2)(1), and finally divide the former by the latter. As pointed out in the 
problem introduction, this can lead to overflows, even when the final result fits in the 
integer word size. 

It is tempting to proceed by pairing terms in the numerator and denominator that 
have common factors and cancel them out. This approach is unsatisfactory because 
of the need to factor numbers, which itself is challenging. 

A better approach is to avoid multiplications and divisions entirely. Fundamen¬ 
tally, the binomial coefficient counts the number of subsets of size A: in a set of size 
n. We could enumerate A:-sized subsets of {0,1,2,..., n - 1) sets using recursion, as in 
Solution 16.5 on Page 291. The idea is as follows. Consider the nth element in the 
initial set. A subset of size k will either contain this element, or not contain it. This is 
the basis for a recursive enumeration—find all subsets of size k- 1 amongst the first 
n - 1 elements and add the nth element into these sets, and then find all subsets of 
size k amongst the first n - 1 elements. The union of these two subsets is all subsets 
of size k. 
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However, since all we care about is the number of such subsets, we can do much 
better complexity-wise. The recursive enumeration also implies that the binomial 
coefficient must satisfy the following formula: 



This identity yields a straightforward recursion for (”). The base cases are Q and 
Q, both of which are 1. The individual results from the subcalls are 32-bit integers 
and if (”) can be represented by a 32-bit integer, they can too, so it is not possible for 
intermediate results to overflow. 

For example, (*) = (*) + ({). Expanding (*), we get (*) = $ + (*). Expanding (*), we 
get ( 2 ) = ( 2 ) + (^). Note that ( 2 ) is a base case, returning 1. Continuing this way, we get 
( 2 ), which is 6 and (*), which is 4, so (Sj) = 6 + 4 = 10. 

Naively implemented, the above recursion will have repeated subcalls with iden¬ 
tical arguments and exponential time complexity. This can be avoided by caching 
intermediate results. 

public static int computeBinomialCoefficient(int n, int k) { 
return computeXChooseY(n, k, new int[n + 1][k + 1]); 

} 

private static int computeXChooseY(int x, int y, int[][] xChooseY) { 
if (y == ® || x == y) { 
return 1; 

} 

if (xChooseY[x][y] == ®) { 

int withoutY = computeXChooseY(x - 1, y, xChooseY); 
int withY = computeXChooseY(x - 1, y - 1, xChooseY); 
xChooseY[x][y] = withoutY + withY; 

} 

return xChooseY[x][y]; 


The number of subproblems is 0(nk) and once ( n ~ l ) and (”“J) are known, (”) can be 
computed in 0(1) time, yielding an 0(nk) time complexity. The space complexity is 
also 0(nky, it can easily be reduced to 0(k). 


17.5 Search for a sequence in a 2D array 

Suppose you are given a 2D array of integers (the "grid"), and a ID array of integers 
(the "pattern"). We say the pattern occurs in the grid if it is possible to start from some 
entry in the grid and traverse adjacent entries in the order specified by the pattern 
till all entries in the pattern have been visited. The entries adjacent to an entry are 
the ones directly above, below, to the left, and to the right, assuming they exist. For 
example, the entries adjacent to (3,4) are (3,3), (3,5), (4,4) and (5,4). It is acceptable 
to visit an entry in the grid more than once. 
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As an example, if the grid is 

[1 2 3' 

3 4 5 

5 6 7 

and the pattern is (1,3,4,6), then the pattern occurs in the grid—consider the entries 
((0,0), (1,0), (1,1), (2,1)). However, (1,2,3,4) does not occur in the grid. 

Write a program that takes as arguments a 2D array and a ID array, and checks 
whether the ID array occurs in the 2D array. 

Hint: Start with length 1 prefixes of the ID array, then move on to length 2,3,... prefixes. 

Solution: A brute-force approach might be to enumerate all ID subarrays of the 2D 
subarray. This has very high time complexity, since there are many possible subarrays. 
Its inefficiency stems from not using the target ID subarray to guide the search. 

Let the 2D array be A, and the ID array be S. Here is a guided way to search for 
matches. Let's say we have a suffix of S to match, and a starting point to match from. 
If the suffix is empty, we are done. Otherwise, for the suffix to occur from the starting 
point, the first entry of the suffix must equal the entry at the starting point, and the 
remainder of the suffix must occur starting at a point adjacent to the starting point. 

For example, when searching for (1,3,4,6) starting at (0,0), since we match 1 with 
A[0][0], we would continue searching for (3,4,6) from (0,1) (which fails immediately, 
since A[0][1] =£ 3) and from (1,0). Since A[0][1] = 3, we would continue searching 
for (4,6) from (0, l)'s neighbors, i.e., from (0,0) (which fails immediately), then from 
(1,1) (which eventually leads to success). 

In the program below, we cache intermediate results to avoid repeated calls to the 
recursion with identical arguments. 


public static boolean isPatternContainedInGrid(List<List<Integer» grid, 

Listdnteger> pattern) { 

// Each entry in previousAttempts is a point in the grid and suffix of 
// pattern (identified by its offset). Presence in previousAttempts 
// indicates the suffix is not contained in the grid starting from that 
// point . 

Set<Attempt> previousAttempts = new HashSet<>(); 
for (int i = ®; i < grid.size(); ++i) { 

for (int j = ®; j < grid.get(i).size () ; ++j) { 

if (isPatternSuffixContainedStartingAtXY(grid, i, j, pattern, ®, 

previousAttempts)) { 


return true; 


} 

} 

} 

return false; 


} 


private static boolean isPatternSuffixContainedStartingAtXY( 

List <List dnteger >> grid, int x, int y, List dnteger > pattern, int offset, 
Set<Attempt> previousAttempts) { 
if (pattern.size() == offset) { 

// Nothing left to complete. 
return true; 
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} 

// Check if (x, y) lies outside the grid. 

if (x < © || x >= grid.sizeO || y < © || y >= grid. get (x) . size () 

|| previousAttempts.contains(new Attempt(x, y, offset))) { 

return false; 

} 

if (grid.get(x).get(y).equals(pattern.get(offset)) 

&<& (isPatternSuffixContainedStartingAtXY(grid, x - 1, y, pattern, 

offset + 1, previousAttempts) 
|| isPatternSuffixContainedStartingAtXY( 

grid, x + 1, y, pattern, offset + 1, previousAttempts) 

|| isPatternSuffixContainedStartingAtXY( 

grid, x, y - 1, pattern, offset + 1, previousAttempts) 

|| isPatternSuffixContainedStartingAtXY( 

grid, x, y + 1, pattern, offset + 1, previousAttempts))) { 

return true; 

} 

previousAttempts.add(new Attempt(x, y, offset)); 

return false; 


The complexity is 0(nm\S\), where n and m are the dimensions of A —we do a constant 
amount of work within each call to the match function, except for the recursive calls, 
and the number of calls is not more than the number of entries in the 2D array. 

Variant: Solve the same problem when you cannot visit an entry in A more than once. 

Variant: Enumerate all solutions when you cannot visit an entry in A more than once. 


17.6 The knapsack problem 

A thief breaks into a clock store. Each clock has a weight and a value, which are known 
to the thief. His knapsack cannot hold more than a specified combined weight. His 
intention is to take clocks whose total value is maximum subject to the knapsack's 
weight constraint. 

His problem is illustrated in Figure 17.8 on the next page. If the knapsack can 
hold at most 130 ounces, he cannot take all the clocks. If he greedily chooses clocks, 
in decreasing order of value-to-weight ratio, he will choose P, H, O, B, I, and L in that 
order for a total value of $669. However, {H, /, 0} is the optimum selection, yielding 
a total value of $695. 

Write a program for the knapsack problem that selects a subset of items that has 
maximum value and satisfies the weight constraint. All items have integer weights 
and values. Return the value of the subset. 

Hint: Greedy approaches are doomed. 

Solution: Greedy strategies such as picking the most valuable clock, or picking the 
clock with maximum value-to-weight ratio, do not always give the optimum solution. 

We can always get the optimum solution by considering all subsets, e.g., using 
Solution 16.4 on Page 290. This has exponential time complexity in the number of 
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J, $120, 30 oz. /, $320, 65 oz. K, $75,75 oz. 



Figure 17.8: A clock store. 


clocks. However, brute-force enumeration is wasteful because it ignores the weight 
constraint. For example, no subset that includes Clocks F and G can satisfy the weight 
constraint. 

The better approach is to simultaneously consider the weight constraint. For 
example, what is the optimum solution if a given clock is chosen, and what is the 
optimum solution if that clock is not chosen? Each of these can be solved recursively 
with the implied weight constraint. For the given example, if we choose the Clock A, 
we need to find the maximum value of clocks from Clocks B-P with a capacity of 
130 - 20 and add $65 to that value. If we do not choose Clock A, we need to find the 
maximum value of clocks from Clocks B-P with a capacity of 130. The larger of these 
two values is the optimum solution. 

More formally, let the clocks be numbered from 0 to n - 1, with the weight and the 
value of the ith clock denoted by zvi and u,-. Denote by V[z][a;] the optimum solution 
when we are restricted to Clocks 0,1,2,...,/- 1 and can carry zv weight. Then V'[f] [w] 
satisfies the following recurrence: 


V[i][zv] = 


max ( V[i - 1 ][w\, V[i - 1 ][zv- zui] + z , 
V[i- l][zv], 


if zui < zu; 
otherwise. 


We take i = 0 or w = 0 as bases cases—for these, V^zy] = 0. 

We demonstrate the above algorithm in Figure 17.9 on the next page. Suppose 
there are four items, sD, ®, whose value and weight are given in Figure 17.9(a) 
on the facing page. Then the corresponding V table is given in Figure 17.9(b) on the 
next page. As an example, the extreme lower right entry is the maximum of the entry 
above (70) and 30 plus the entry for sD, 0, ® with capacity 5-2 (50), i.e., 80. 

private static class Item { 
public Integer weight; 
public Integer value; 


public Item(Integer weight, Integer value) { 
this.weight = weight; 
this.value = value; 

} 

} 
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0 1 2 3 4 5 


Item 

Value 

Weight 

$ 

$60 

5 oz. 

i 

$50 

3 oz. 


$70 

4 oz. 

tS 

$30 

2 oz. 


(a) Value-weight table. 



0 

0 

0 

0 

0 

60 

50S 

0 

0 

0 

50 

50 

60 

$0® 

0 

0 

0 

50 

70 

70 


0 

0 

30 

50 

70 

80 


(b) Knapsack table for the items in (a). The 
columns correspond to capacities from 0 to 5. 


Figure 17.9: A small example of the knapsack problem. The knapsackcapacity is 5 oz. 


public static int optimumSubjectToCapacity(List<Item> items, int capacity) { 

// V[i][j] holds the optimum value when we choose from items[Q : i] and have 
// a capacity of j. 

int[][] V = new int[items.size()][capacity + 1]; 
for (int[] v : V) { 

Arrays.fill(v, -1); 

} 

return optimumSubjectToItemAndCapacity(items, items.size() - 1, capacity, 


// Returns the optimum value when we choose from items[9 : k] and have a 
// capacity of available_capacity . 

private static int optimumSubjectToItemAndCapacity(List<Item> items, int k, 

int availableCapacity, 
int[][] V) { 

if (k < ®) { 

// No items can be chosen. 
return ®; 

} 

if (V[k][availableCapacity] == -1) { 
int withoutCurrltem 

= optimumSubjectToItemAndCapacity(items, k - 1, availableCapacity, V); 
int withCurrltem 

= availableCapacity < items.get(k).weight 
? ® 

: items.get(k).value 

+ optimumSubj ectToItemAndCapacity( 
items , k - 1, 

availableCapacity - items.get(k).weight, V); 

V[k][availableCapacity] = Math.max(withoutCurrltem, withCurrltem); 

} 

return V[k][availableCapacity] ; 


The algorithm computes V[n - l][w] in 0(nw) time, and uses 0(nzv) space. 

Variant: Solve the same problem using 0(zv) space. 

Variant: Solve the same problem using 0(C) space, where C is the number of 
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weights between 0 and zv that can be achieved. For example, if the weights 
are 100, 200, 200, 500, and w - 853, then C - 9, corresponding to the weights 
0,100,200,300,400,500,600,700,800. 

Variant: Solve the fractional knapsack problem. In this formulation, the thief can take 
a fractional part of an item, e.g., by breaking it. Assume the value of a fraction of an 
item is that fraction times the value of the item. 

Variant: In the "divide-the-spoils-fairly" problem, two thieves who have successfully 
completed a burglary want to know how to divide the stolen items into two groups 
such that the difference between the value of these two groups is minimized. For 
example, they may have stolen the clocks in Figure 17.8 on Page 318, and would like 
to divide the clocks between them so that the difference of the dollar value of the two 
sets is minimized. For this instance, an optimum split is \A, G,],M, 0,P} to one thief 
and the remaining to the other thief. The first set has value $1179, and the second has 
value $1180. An equal split is impossible, since the sum of the values of all the clocks 
is odd. Write a program to solve the divide-the-spoils-fairly problem. 

Variant: Solve the divide-the-spoils-fairly problem with the additional constraint that 
the thieves have the same number of items. 

Variant: The US President is elected by the members of the Electoral College. The 
number of electors per state and Washington, D.C., are given in Table 17.1. All electors 
from each state as well as Washington, D.C., cast their vote for the same candidate. 
Write a program to determine if a tie is possible in a presidential election with two 
candidates. 


Table 17.1: Electoral college votes. 


State 

Electors 

State 

Electors 

State 

Electors 

Alabama 

9 

Louisiana 

8 

Ohio 

18 

Alaska 

3 

Maine 

4 

Oklahoma 

7 

Arizona 

11 

Maryland 

10 

Oregon 

7 

Arkansas 

6 

Massachusetts 

11 

Pennsylvania 

20 

California 

55 

Michigan 

16 

Rhode Island 

4 

Colorado 

9 

Minnesota 

10 

South Carolina 

9 

Connecticut 

7 

Mississippi 

6 

South Dakota 

3 

Delaware 

3 

Missouri 

10 

Tennessee 

11 

Florida 

29 

Montana 

3 

Texas 

38 

Georgia 

16 

Nebraska 

5 

Utah 

6 

Hawaii 

4 

Nevada 

6 

Vermont 

3 

Idaho 

4 

New Hampshire 

4 

Virginia 

13 

Illinois 

20 

New Jersey 

14 

Washington 

12 

Indiana 

11 

New Mexico 

5 

West Virginia 

5 

Iowa 

6 

New York 

29 

Wisconsin 

10 

Kansas 

6 

North Carolina 

15 

Wyoming 

3 

Kentucky 

8 

North Dakota 

3 

Washington, D.C. 

3 


17.7 The bedbathandbeyond.com problem 

Suppose you are designing a search engine. In addition to getting keywords from 
a page's content, you would like to get keywords from Uniform Resource Locators 
(URLs). For example, bedbathandbeyond.com yields the keywords "bed, bath, beyond. 
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bat, hand": the first two coming from the decomposition "bed bath beyond" and the 
latter two coming from the decomposition "bed bat hand beyond". 

Given a dictionary, i.e., a set of strings, and a name, design an efficient algorithm that 
checks whether the name is the concatenation of a sequence of dictionary words. If 
such a concatenation exists, return it. A dictionary word may appear more than once 
in the sequence, e.g., "a", "man", "a", "plan", "a", "canal" is a valid sequence for 
"amanaplanacanal". 

Hint: Solve the generalized problem, i.e., determine for each prefix of the name whether it is the 
concatenation of dictionary words. 

Solution: The natural approach is to use recursion, i.e., find dictionary words that 
begin the name, and solve the problem recursively on the remainder of the name. 
Implemented naively, this approach has very high time complexity on some input 
cases, e.g., if the name is N repetitions of "AB" followed by "C" and the dictionary 
words are "A", "B", and "AB" the time complexity is exponential in N. For example, 
for the string "ABABC", then we recurse on the substring "ABC" twice (once from 
the sequence "A", "B" and once from "AB"). 

The solution is straightforward—cache intermediate results. The cache keys are 
prefixes of the string. The corresponding value is a Boolean denoting whether the 
prefix can be decomposed into a sequence of valid words. 

It's easy to determine if a string is a valid word—we simply store the dictionary 
in a hash table. A prefix of the given string can be decomposed into a sequence of 
dictionary words exactly if it is a dictionary word, or there exists a shorter prefix 
which can be decomposed into a sequence of dictionary words and the difference of 
the shorter prefix and the current prefix is a dictionary word. 

For example, for "amanaplanacanal": 

1. the prefix "a" has a valid decomposition (since "a" is a dictionary word), 

2. the prefix "am" has a valid decomposition (since "am" is a dictionary word), 

3. the prefix "ama" has a valid decomposition (since "a" has a valid decomposition, 
and "am" is a dictionary word), and 

4. "aman" has a valid decomposition (since "am" has a valid decomposition and 
"an" is a dictionary word). 

Skipping ahead, 

5. "amanapl" does not have a valid decomposition (since none of "1", "pi", apl", 
etc. are dictionary words), and 

6. "amanapla" does not have a valid decomposition (since the only dictionary 
word ending the string is "a" and "amanapl" does not have a valid decompo¬ 
sition). 

The algorithm tells us if we can break a given string into dictionary words, but 
does not yield the words themselves. We can obtain the words with a little more 
book-keeping. Specifically, if a prefix has a valid decomposition, we record the length 
of the last dictionary word in the decomposition. 

public static List<String> decomposelntoDictionaryWords( 

String domain, Set<String> dictionary) { 
int[] lastLength = new int[domain.length()]; 
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Arrays.fill(lastLength, -1); 

// When the algorithm finishes, lastLength[i] != -1 indicates 

// domain.substring(Q, i + 1) has a valid decomposition, and the length of 

// the last string in the decomposition will be lastLength[i]. 

for (int i = ®; i < domain. length () ; ++i) { 

// If domain.substring(&, i + 1) is a valid word, set lastLength[i] to the 
// length of that word. 

if (dietionary.contains(domain.substring(®, i + 1))) { 
lastLength[i] = i + 1; 

} 

// If lastLength[i] = -1 look for j < i such that domain.substring(Q, j + 
// 1) has a valid decomposition and domain.substring(j + 1, i + 1) is a 
// dictionary word. If so, record the length of that word in 
// lastLength[i]. 
if (lastLength[i] == -1) { 

for (int j = ®; j < i; ++j) { 
if (lastLength[j] != -1 

&& dietionary.contains(domain.substring(j + 1, i + 1))) { 
lastLength[i] = i - j; 

break; 

} 

} 

} 

} 

List<String> decompositions = new ArrayList<>(); 
if (lastLength[lastLength.length - 1] != -1) { 

// domain can be assembled by valid words. 
int idx = domain.length() - 1; 
while (idx >= ®) { 
decompositions.add( 

domain.substring(idx + 1 - lastLength[idx], idx + 1)); 
idx -= lastLength[idx]; 

} 

Collections.reverse(decompositions); 

} 

return decompositions; 


Let n be the length of the input string s. For each k < n we check for each j < k 
whether the substring s[j +1 : k] is a dictionary word, and each such check requires 
0(k - j) time. This implies the time complexity is 0(n 3 ). We can improve the time 
complexity as follows. Let W be the length of the longest dictionary word. We can 
restrict j to range from k - W to k - 1 without losing any decompositions, so the time 
complexity improves to 0(n 2 W). 

If we want all possible decompositions, we can store all possible values of j that 
gives us a correct break with each position. Note that the number of possible decom¬ 
positions can be exponential here. This is illustrated by the string "itsitsitsits... ". 

Variant: Palindromic decompositions were described in Problem 16.7 on Page 293. 
Observe every string s has at least one palindromic decomposition, which is the trivial 
one consisting of the individual characters. For example, if s is "0204451881" then 
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"0", "2", "0", "4", "4", "5", "1", "8", "8", "1" is such a trivial decomposition. The 
minimum decomposition of s is "020", "44", "5", "1881". How would you compute a 
palindromic decomposition of a string s that uses a minimum number of substrings? 


17.8 Find the minimum weight path in a triangle 

A sequence of integer arrays in which the nth array consists of n entries naturally 
corresponds to a triangle of numbers. See Figure 17.10 for an example. 
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Figure 17.10: A number triangle. 


Define a path in the triangle to be a sequence of entries in the triangle in which 
adjacent entries in the sequence correspond to entries that are adjacent in the triangle. 
The path must start at the top, descend the triangle continuously, and end with an 
entry on the bottom row. The weight of a path is the sum of the entries. 

Write a program that takes as input a triangle of numbers and returns the weight of 
a minimum weight path. For example, the minimum weight path for the number 
triangle in Figure 17.10 is shown in bold face, and its weight is 15. 

Hint: What property does the prefix of a minimum weight path have? 

Solution: A brute-force approach is to enumerate all possible paths. Although these 
paths are quite easy to enumerate, there are 2” -1 such paths in a number triangle with 
n rows. 

A far better way is to consider entries in the ith row. For any such entry, if you 
look at the minimum weight path ending at it, the part of the path that ends at the 
previous row must also be a minimum weight path. This gives us a DP solution. We 
iteratively compute the minimum weight of a path ending at each entry in Row i. 
Since after we complete processing Row i, we do not need the results for Row / — 1 to 
process Row i + 1, we can reuse storage. 

public static int minimumPathTotal(ListcListdnteger>> triangle) { 
if (triangle.isEmpty()) { 
return Q; 

} 

// As we iterate, prevRow stores the minimum path sum to each entry in 
// triangle . get(i - 1). 

Listdnteger> prevRow = new ArrayList<>(triangle.get(®)); 
for (int i = 1; i < triangle. size () ; ++i) { 
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// Stores the minimum path sum to each entry in triangle.get(i). 
Listdnteger > currRow = new ArrayList <>(triangle . get (i) ) ; 

// For the first element. 

currRow.set(©, currRow.get(Q) + prevRow.get(®)) ; 
for (int j = 1; j < currRow. size () - 1; ++j) { 
currRow.set( 

j, currRow.get(j) + Math.min(prevRow.get(j - 1), prevRow.get(j))); 

} 

// For the last element 

currRow.set(currRow.size() - 1, currRow.get(currRow.size() - 1) 

+ prevRow.get(prevRow.size() - 1)); 


prevRow = currRow; 

} 

return Collections.min(prevRow); 


The time spent per element is 0(1) and there are 1 + 2 H-1 -n - n(n + l)/2 elements, 

implying an 0(n 2 ) time complexity. The space complexity is 0(n). 


17.9 Pick up coins for maximum gain 

In the pick-up-coins game, an even number of coins are placed in a line, as in Fig¬ 
ure 17.11. Two players take turns at choosing one coin each—they can only choose 
from the two coins at the ends of the line. The game ends when all the coins have 
been picked up. The player whose coins have the higher total value wins. A player 
cannot pass his turn. 



Figure 17.11: A row of coins. 


Design an efficient algorithm for computing the maximum total value for the starting 
player in the pick-up-coins game. 

Hint: Relate the best play for the first player to the best play for the second player. 

Solution: First of all, note that greedily selecting the maximum of the two end coins 
does not yield the best solution. If the coins are 5,25,10,1, if the first player is greedy 
and chooses 5, the second player can pick up the 25. Now the first player will choose 
the 10, and the second player gets the 1, so the first player has a total of 15 and the 
second player has a total of 26. A better move for the first player is picking up 1. Now 
the second player is forced to expose the 25, so the first player will achieve 26. 

The drawback of greedy selection is that it does not consider the opportunities 
created for the second player. Intuitively, the first player wants to balance selecting 
high coins with minimizing the coins available to the second player. 

The second player is assumed to play the best move he possibly can. Therefore, 
the second player will choose the coin that maximizes his revenue. Call the sum of the 
coins selected by a player his revenue. Let R(a, h) be the maximum revenue a player 
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can get when it is his turn to play, and the coins remaining on the table are at indices 
a to b, inclusive. Let C be an array representing the line of coins, i.e., C[i] is the value 
of the zth coin. If the first player selects the coin at a, since the second player plays 
optimally, the first player will end up with a total revenue of C[a\+S(a+ 1, b)-R(a+ 1, b ), 
where S(a , b) is the sum of the coins from positions a to b, inclusive. If he selects the 
coin at b, he will end up with a total revenue of C[b] + S(a, b - 1) - R(a, b - 1). Since 
the first player wants to maximize revenue, he chooses the greater of the two, i.e., 
R(a , b) = max(C[fl] + S(a +1, b) - R(a +1, b), C[b] + S{a, b- 1) - R{a, b - 1)). This recursion 
for R can be solved easily using DP. 

Now we present a slightly different recurrence for R. Since the second player seeks 
to maximize his revenue, and the total revenue is a constant, it is equivalent for the 
the second player to move so as to minimize the first player's revenue. Therefore, 
R(a, b) satisfies the following equations: 


r 

f 

C[a] + min 

{ R(a + 2,b), \ 

\ 


K R(a+l,b-l) / 

/ 

max 

C[b] + min j 

R(a + l,b-l), 

\ R(a,b-2) 

), 

0, 





if a < b) 


otherwise. 


In essence, the strategy is to minimize the maximum revenue the opponent can gain. 
The benefit of this "min-max" recurrence for R{a, b), compared to our first formulation, 
is that it does not require computing S(« + 1, b) and S(a, b - 1). 

For the coins (10,25,5,1,10,5), the optimum revenue for the firstplayer is 31. Some 
of subproblems encountered include computing the optimum revenue for (10,25) 
(which is 25), (5,1) (which is 5), and (5,1,10,5) (which is 15). 

For the coins in Figure 17.11 on the facing page, the maximum revenue for the 
first player is 140tf, i.e., whatever strategy the second player uses, the first player can 
guarantee a revenue of at least 140tf. In contrast, if both players always pick the more 
valuable of the two coins available to them, the first player will get only 120<2. 

In the program below, we solve for R using DP. 


public static int pickUpCoins(List<Integer> coins) { 

return computeMaximumRevenueForRange(coins, ®, coins.size() - 1, 

new int [coins . size () ] [coins. sizeQ]) ; 


} 


private static int computeMaximumRevenueForRange( 

Listdnteger> coins, int a, int b, int[][] maximumRevenueForRange) { 
if (a > b) { 

// No coins left. 
return Q; 


if (maximumRevenueForRange[a][b] == ®) { 
int maximumRevenueA 
= coins.get(a) 

+ Math.min(computeMaximumRevenueForRange(coins, a + 2, b, 

maximumRevenueForRange), 

computeMaximumRevenueForRange(coins, a + 1, b - 1, 
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maximumRevenueForRange)); 


int maximumRevenueB 
= coins.get(b) 

+ Math.min(computeMaximumRevenueForRange(coins, a + 1, b - 1, 

maximumRevenueForRange), 
computeMaximumRevenueForRange(coins, a, b - 2, 

maximumRevenueForRange)); 

maximumRevenueForRange[a][b] = Math.max(maximumRevenueA, maximumRevenueB); 

} 

return maximumRevenueForRange[a][b]; 

} 


There are 0(n 2 ) possible arguments for R(a, b ), where n is the number of coins, and 
the time spent to compute R from previously computed values is 0( 1). Hence, R can 
be computed in 0(n 2 ) time. 


17.10 Count the number of moves to climb stairs 

You are climbing stairs. You can advance 1 to A: steps at a time. Your destination is 
exactly n steps up. 

Write a program which takes as inputs n and k and returns the number of ways in 
which you can get to your destination. For example, if n = 4 and k = 2, there are five 
ways in which to get to the destination: 

• four single stair advances, 

• two single stair advances followed by a double stair advance, 

• a single stair advance followed by a double stair advance followed by a single 
stair advance, 

• a double stair advance followed by two single stairs advances, and 

• two double stair advances. 

Hint: How many ways are there in which you can take the last step? 

Solution: A brute-force enumerative solution does not make sense, since there are an 
exponential number of possibilities to consider. 

Since the first advance can be one step, two steps,..., k steps, and all of these lead 
to different ways to get to the top, we can write the following equation for the number 
of steps F{n, k): 

it 

F(n,k) = Y j F(n-i,k) 

i =1 

For the working example, F( 4,2) = F(4 - 2,2) + F(4 - 1,2). Recursing, F(4 - 2,2) = 
F(4 - 2 - 2,2) + F(4 -2-1,2). Both F(0,2) and F(l, 2) are base-cases, with a value of 1, 
so F(4 - 2,2) = 2. Continuing with F(4 -1,2), F(4 -1,2) = F(4 -1 - 2,2) + F(4 -1 -1,2). 
The first term is a base-case, with a value of 1. The second term has already been 
computed—its value is 2. Therefore, F(4 — 1,2) = 3, and F(4,2) = 3 + 2. 

In the program below, we cache values of F(i,k), 0 < i < n to improve time 
complexity. 
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public static int numberOfWaysToTop(int top, int maximumStep) { 

return computeNumberOfWaysToH(top, maximumStep, new int[top + 1]); 

} 

private static int computeNumberOfWaysToH(int n, int maximumStep, 

int[] numberOfWaysToH) { 

if (n <= 1) { 

return 1; 

} 

if (numberOfWaysToH[n] == ®) { 

for (int i = 1; i <= maximumStep <&& n - i >= 8; ++i) { 
numberOfWaysToH[n] 

+= computeNumberOfWaysToH(n - i, maximumStep, numberOfWaysToH); 

} 

} 

return numberOfWaysToH[n]; 


We take 0(k) time to fill in each entry, so the total time complexity is 0(kn). The space 
complexity is 0(n). 


17.11 The pretty printing problem 

Consider the problem of laying out text using a fixed width font. Each line can hold 
no more than a fixed number of characters. Words on a line are to be separated by 
exactly one blank. Therefore, we may be left with whitespace at the end of a line 
(since the next word will not fit in the remaining space). This whitespace is visually 
unappealing. 

Define the messiness of the end-of-line whitespace as follows. The messiness of a 
single line ending with b blank characters is b 2 . The total messiness of a sequence of 
lines is the sum of the messinesses of all the lines. A sequence of words can be split 
across lines in different ways with different messiness, as illustrated in Figure 17.12. 


I have inserted a large number of 
new examples from the papers for the 
Mathematical Tripos during the last,., 
twenty years, which should be useful 
to Cambridge students 

(a) Messiness = 3 2 + 0 2 + l 2 + 0 2 + 14 2 = 206. 


I have inserted a large number uuuuuu 
of new examples from the papers^^^ 
for the Mathematical Tripos during^ 
the last twenty years, which should^ 
be useful to Cambridge students >LJLJLJLJ 

(b) Messiness = 6 2 + 5 2 + 2 2 + l 2 + 4 2 = 82. 


Figure 17.12: Two layouts for the same sequence of words; the line length L is 36. 


Given text, i.e., a string of words separated by single blanks, decompose the text into 
lines such that no word is split across lines and the messiness of the decomposition 
is minimized. Each line can hold no more than a specified number of characters. 

Hint: Focus on the last word and the last line. 

Solution: A greedy approach is to fit as many words as possible in each line. However, 
some experimentation shows this is suboptimal. See Figure 17.13 for an example. In 
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essence, the greedy algorithm does not spread words uniformly across the lines, 
which is the key requirement of the problem. Adding a new word may necessitate 
moving earlier words. 


0 12 3 4 


0 12 3 4 


Line 1 
Line 2 

(a) Greedy placement: Line 1 has a messiness of 0 2 , 
and Line 2 has a messiness of 4 2 . 



Line 1 
Line 2 

(b) Optimum placement: Line 1 has a messiness of 
2 2 , and Line 2 has a messiness of 2 2 . 



Figure 17.13: Assuming the text is “a b c d” and the line length is 5, placing words greedily results in a 
total messiness of 16, as seen in (a), whereas the optimum placement has a total messiness of 8, as 
seen in (b). 


Suppose we want to find the optimum placement for the zth word. As we have just 
seen, we cannot trivially extend the optimum placement for the first i - 1 words to an 
optimum placement for the zth word. Put another way, the placement that results by 
removing the zth word from an optimum placement for the first z words is not always 
an optimum placement for the first i— 1 words. However, what we can say is that if 
in the optimum placement for the zth word the last line consists of words ;,; +1,..., z, 
then in this placement, the first j— 1 words must be placed optimally. 

In an optimum placement of the first z words, the last line consists of some subset 
of words ending in the zth word. Furthermore, since the first z words are assumed to 
be optimally placed, the placement of words on the lines prior to the last one must be 
optimum. Therefore, we can write a recursive formula for the minimum messiness, 
M(z), when placing the first z words. Specifically, M(z), equals min ; < z /(;, z) + M(j - 1), 
where /(;, z) is the messiness of a single line consisting of words j to z inclusive. 

We give an example of this approach in Figure 17.14 on the next page. The optimum 
placement of "aaa bbb c d ee" is shown in Figure 17.14(a) on the facing page. The 
optimum placement of "aaa bbb c d ee ff" is shown in Figure 17.14(b) on the next 
page. 

To determine the optimum placement for "aaa bbb c d ee ff ggggggg", we consider 
two cases—the final line is "ff ggggggg", and the final line is "ggggggg"- (No more 
cases are possible, since we cannot fit "ee ff ggggggg" on a single line.) 

If the final line is "ff ggggggg", then the "aaa bb c d ee" must be placed as in 
Figure 17.14(a) on the facing page. If the final line is "ggggggg", then "aaa bbb c 
d ee ff" must be placed as in Figure 17.14(b) on the next page. These two cases are 
shown in Figure 17.14(c) on the facing page and Figure 17.14(d) on the next page, 
respectively. Comparing the two, we see Figure 17.14(c) on the facing page has the 
lower messiness and is therefore optimum. 

The recursive computation has exponential complexity, because it visits identical 
subproblems repeatedly. The solution is to cache the values for M. 

public static int minimumMessiness(List<String> words, int lineLength) { 

// minimumMessiness[i] is the minimum messiness when placing words[9 : i]. 
int[] minimumMessiness = new int[words.size()]; 
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(a) Optimum placement for “aaa bbb c d ee”. (b) Optimum placement for “aaa bbb c d ee ff”. 
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(c) Optimum placement for “aaa bbb c d ee ff 
ggggggg” when the final line is “ggggggg”. 
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(d) Optimum placement for “aaa bbb c d ee ff 
ggggggg” when the final line is “ff ggggggg”. 


Figure 17.14: Solving the pretty printing problem for text “aaa bb c d ee ff ggggggg” and line length 11. 


Arrays.fill(minimumMessiness, Integer.MAX_VALUE); 
int numRemainingBlanks = lineLength - words.get(®).length(); 
minimumMessiness[®] = numRemainingBlanks * numRemainingBlanks; 
for (int i = 1; i < words.size(); ++i) { 

numRemainingBlanks = lineLength - words.get(i).length(); 
minimumMessiness[i] 

= minimumMessiness[i - 1] + numRemainingBlanks * numRemainingBlanks; 
// Try adding words.get(i - 1), words.get(i -2), ... 

for (int j = i - 1; j >= ®; --j) { 

numRemainingBlanks -= (words.get(j).length() + 1); 
if (numRemainingBlanks < ®) { 

// Not enough space to add more words. 
break; 

} 

int firstJMessiness =j-l<®?®: minimumMessiness[j - 1]; 
int currentLineMessiness = numRemainingBlanks * numRemainingBlanks; 
minimumMessiness[i] = Math.min(minimumMessiness[i], 

firstJMessiness + currentLineMessiness); 

} 

} 

return minimumMessiness[words.size() - 1]; 


Let L be the line length. Then there can certainly be no more than L words on a line, 
so the amount of time spent processing each word is 0(L). Therefore, if there are n 
words, the time complexity is 0(nL). The space complexity is 0(n) for the cache. 

Variant: Solve the same problem when the messiness is the sum of the messinesses 
of all but the last line. 

Variant: Suppose the messiness of a line ending with b blank characters is defined 
to be b. Can you solve the messiness minimization problem in 0(n) time and 0(1) 
space? 
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17.12 Find the longest nondecreasing subsequence 


The problem of finding the longest nondecreasing subsequence in a sequence of inte¬ 
gers has implications to many disciplines, including string matching and analyzing 
card games. As a concrete instance, the length of a longest nondecreasing subse¬ 
quence for the array in Figure 17.15 is 4. There are multiple longest nondecreasing 
subsequences, e.g., (0,4,10,14) and (0,2,6,9). Note that elements of non-decreasing 
subsequence are not required to immediately follow each other in the original se¬ 
quence. 


0 

8 

4 

12 

2 

10 

6 

14 

1 

9 

A[ 0] 

A[ 1] 

A[ 2] 

A[3] 

A[ 4] 

A[5] 

A[6] 

A[7] 

A[8] 

A[9] 


Figure 17.15: An array whose longest nondecreasing subsequences are of length 4. 


Write a program that takes as input an array of numbers and returns the length of a 
longest nondecreasing subsequence in the array. 

Hint: Express the longest nondecreasing subsequence ending at an entry in terms of the longest 
nondecreasing subsequence appearing in the subarray consisting of preceding elements. 

Solution: A brute-force approach would be to enumerate all possible subsequences, 
testing each one for being nondecreasing. Since there are T subsequences of an array 
of length n, the time complexity would be huge. Some heuristic pruning can be 
applied, but the program grows very cumbersome. 

If we have processed the initial set of entries of the input array, intuitively this 
should help us when processing the next entry. For the given example, if we know 
the lengths of the longest nondecreasing subsequences that end at Indices 0,1,..., 5 
and we want to know the longest nondecreasing subsequence that ends at Index 6, 
we simply look for the longest subsequence ending at an entry in A[0 : 5] whose value 
is less than or equal to A[ 6]. For Index 6 in the example in Figure 17.15, there are two 
such longest subsequences, namely the ones ending at Index 2 and Index 4, both of 
which yield nondecreasing subsequences of length 3 ending at A[ 6]. 

Generalizing, define L[i] to be the length of the longest nondecreasing subsequence 
of A that ends at and includes Index i. As an example, for the array in Figure 17.15, 
L[6] = 3. The longest nondecreasing subsequence that ends at Index i is of length 1 
(if A[i] is smaller than all preceding entries) or has some element, say at Index j, as 
its penultimate entry, in which case the subsequence restricted to A[0 : /] must be the 
longest subsequence ending at Index j. Based on this, L[i] is either 1 (if A[i] is less 
than all previous entries), or 1 + max{L[/]|/ < i and A[j] < A[i]}. 

We can use this relationship to compute L, recursively or iteratively. If we want 
the sequence as well, for each i, in addition to storing the length of the nondecreasing 
sequence ending at i, we store the index of the last element of the subsequence that 
we extended to get the value assigned to L[i]. 

Applying this algorithm to the example in Figure 17.15, we compute L as follows: 

1. L[0] = 1 (since there are no entries before Entry 0) 
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2. L[ 1] = 1 + max(L[0]) = 2 (since A[0] < A[l]) 

3. L[2] = 1 + max(L[0]) = 2 (since A[0] < A[ 2] and A[l] > A[2]) 

4. L[3] = 1 + max(L[0],L[l],L[2]) = 3 (since A[0], A[1] and A[2] < A[3]) 

5. L[4] = 1 + max(L[0]) = 2 (since A[0] < A[4], A[l], A[2] and A[3] > A[4]) 

6. L[5] = 1 + max(L[0],L[l],L[2],L[4]) = 3 (since A[0], A[l], A[2], A[4] < A[5] and 

A[3] > A[5]) 

7. L[6] = 1 + max(L[0],L[2],L[4]) = 3 (since A[0], A[2], A[4] < A[6] and A[l], A[3], 
A[5] > A[6]) 

8. L[7] = l+max(L[0], L[l], L[2], L[3], L[4], L[5], L[6]) = 4 (since A[0],A[1],A[2], A[3], 
A[4],A[5],A[6]<A[7]) 

9. L[8] = 1 + max(L[0]) = 2 (since A[0] < A[8] and A[1],A[2], A[3], A[4], A[5], A[6], 
A[7] > A[8]) 

10. L[9] = 1 + max(L[0],L[l],L[2],L[4],L[6],L[8]) = 4 (since A[0], A[l], A[2], A[4], 
A[6], A[8] < A[9] and A[3], A[5], A[7] > A[9]) 

Therefore the maximum length of a longest nondecreasing subsequence is 4. There 
are multiple longest nondecreasing sequences of length 4, e.g., (0,8,12,14), (0,2,6,9), 
<0,4,6,9). 

Here is an iterative implementation of the algorithm. 


public static int longestNondecreasingSubsequenceLength(List<Integer> A) { 

// maxLength[i] holds the length of the longest nondecreasing subsequence of 
// A[<9 : i]. 

Integer[] maxLength = new Integer[A.size()]; 

Arrays.fill(maxLength, 1); 
for (int i = 1; i < A.sizeO; ++i) { 
for (int j = ®; j < i; ++j) { 
if (A.get(i) >= A.get(j)) { 

maxLength[i] = Math.max(maxLength[i], maxLength[j] + 1); 

} 

> 

} 

return Collections.max(Arrays.asList(maxLength)); 


The time complexity is 0(n 2 ) (each L[i] takes 0(n) time to compute), and the space 
complexity is 0(n) (to store L). 

Variant: Write a program that takes as input an array of numbers and returns a longest 
nondecreasing subsequence in the array 

Variant: Define a sequence of numbers (a 0 ,fli,... to be alternating if a x < 

a i+ 1 for even i and > a i+ \ for odd i. Given an array of numbers A of length n, find a 
longest subsequence (z'o,..., 4-i) such that <A[z'o], A[z‘i],..., A[4-i]) is alternating. 

Variant: Define a sequence of numbers (a§,a\,... ,a n _ i) to be weakly alternating if no 
three consecutive terms in the sequence are increasing or decreasing. Given an 
array of numbers A of length n, find a longest subsequence <4,...,4-i) such that 
<A[z 0 ], A[4], • • •, A[4-i]> is weakly alternating. 

Variant: Define a sequence of numbers (a 0/ a^, ..., a n _ i) to be convex if a t < f for 

1 < i < n — 2. Given an array of numbers A of length n, find a longest subsequence 
<4,..., 4-i) such that <A[4],A[4],..., A[4-i]) is convex. 
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Variant: Define a sequence of numbers {a 0 , ,..., a n _\) to be bitonic if there exists k such 

that ci} < a i+ 1 , for 0 < i < k and > a i+1 , for k < i < n - 1. Given an array of numbers A 
of length n, find a longest subsequence (io, ..., 4-i) such that (A[io], A[i\\,..., A[4-i]) 
is bitonic. 

Variant: Define a sequence of points in the plane to be ascending if each point is above 
and to the right of the previous point. How would you find a maximum ascending 
subset of a set of points in the plane? 

Variant: Compute the longest nondecreasing subsequence in 0(n log n) time. 
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Chapter 


Greedy Algorithms and Invariants 

The intendedfunction of a program, or part of a program, can be 
specified by making general assertions about the values which 
the relevant variables will take after execution of the program. 

— "An Axiomatic Basis for Computer Programming," 
C. A. R. Hoare, 1969 


Greedy algorithms 

A greedy algorithm is an algorithm that computes a solution in steps; at each step 
the algorithm makes a decision that is locally optimum, and it never changes that 
decision. 

The example on Page 38 illustrates how different greedy algorithms for the 
same problem can differ in terms of optimality. As another example, consider 
making change for 48 pence in the old British currency where the coins came in 
30,24,12,6,3, and 1 pence denominations. Suppose our goal is to make change us¬ 
ing the smallest number of coins. The natural greedy algorithm iteratively chooses 
the largest denomination coin that is less than or equal to the amount of change that 
remains to be made. If we try this for 48 pence, we get three coins—30 + 12 + 6. 
However, the optimum answer would be two coins—24 + 24. 

In its most general form, the coin changing problem is NP-hard on Page 41, but 
for some coinages, the greedy algorithm is optimum—e.g., if the denominations are 
of the form {1, r, r 2 , r 3 }. (An ad hoc argument can be applied to show that the greedy 
algorithm is also optimum for US coinage.) The general problem can be solved in 
pseudo-polynomial time using DP in a manner similar to Problem 17.6 on Page 317. 

As another example of how greedy reasoning can fail, consider the following 
problem: Four travelers need to cross a river as quickly as possible in a small boat. 
Only two people can cross at one time. The time to cross the river is dictated by the 
slower person in the boat (if there is just one person, that is his time). The four travelers 
have times of 5,10,20, and 25 minutes. The greedy schedule would entail having the 
two fastest travelers cross initially (10), with the fastest returning (5), picking up the 
faster of the two remaining and crossing again (20), and with the fastest returning for 
the slowest traveler (5 + 25). The total time taken would be 10 + 5 + 20 + 5 + 25 = 65 
minutes. However, a better approach would be for the fastest two to cross (10), with 
the faster traveler returning (5), and then having the two slowest travelers cross (25), 
with the second fastest returning (10) to pick up the fastest traveler (10). The total 
time for this schedule is 10 + 5 +25+ 10 + 10 = 60 minutes. 


333 




Greedy algorithms boot camp 

For US currency, wherein coins take values 1,5,10,25,50,100 cents, the greedy al¬ 
gorithm for making change results in the minimum number of coins. Here is an 
implementation of this algorithm. Note that once it selects the number of coins of 
a particular value, it never changes that selection; this is the hallmark of a greedy 
algorithm. 

public static int changeMaking(int cents) { 
final int[] COINS = (1®®, 5®, 25, 1®, 5, 1}; 
int numCoins = ®; 

for (int i = ®; i < COINS.length; i++) { 
numCoins += cents / COINS[i]; 
cents %= COINS[i] ; 

} 

return numCoins; 


We perform 6 iterations, and each iteration does a constant amount of computation, 
so the time complexity is 0(1). 


A greedy algorithm is often the right choice for an optimization problem where 
there's a natural set of choices to select from. [Problem 18.1] 

It's often easier to conceptualize a greedy algorithm recursively, and then imple¬ 
ment it using iteration for higher performance. [Problem 25.34] 

Even if the greedy approach does not yield an optimum solution, it can give 
insights into the optimum algorithm, or serve as a heuristic. 

Sometimes the right greedy choice is not obvious. [Problem 4.2] 


18.1 Compute an optimum assignment of tasks 

We consider the problem of assigning tasks to workers. Each worker must be assigned 
exactly two tasks. Each task takes a fixed amount of time. Tasks are independent, i.e., 
there are no constraints of the form "Task 4 cannot start before Task 3 is completed." 
Any task can be assigned to any worker. 

We want to assign tasks to workers so as to minimize how long it takes before all 
tasks are completed. For example, if there are 6 tasks whose durations are 5,2,1,6,4,4 
hours, then an optimum assignment is to give the first two tasks (i.e., the tasks with 
duration 5 and 2) to one worker, the next two (1 and 6) to another worker, and the 
last two tasks (4 and 4) to the last worker. For this assignment, all tasks will finish 
after max(5 + 2,1 + 6,4 + 4) = 8 hours. 

Design an algorithm that takes as input a set of tasks and returns an optimum assign¬ 
ment. 

Hint: What additional task should be assigned to the worker who is assigned the longest task? 
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Solution: Simply enumerating all possible sets of pairs of tasks is not feasible—there 
are too many of them. (The precise number assignments is (IJX^X”^ 4 ) *' * ( 2 X 2 ) = 
n\/2 n/1 , where n is the number of tasks.) 

Instead we should look more carefully at the structure of the problem. Extremal 
values are important—the task that takes the longest needs the most help. In partic¬ 
ular, it makes sense to pair the task with longest duration with the task of shortest 
duration. This intuitive observation can be understood by looking at any assignment 
in which a longest task is not paired with a shortest task. By swapping the task that 
the longest task is currently paired with with the shortest task, we get an assignment 
which is at least as good. 

Note that we are not claiming that the time taken for the optimum assignment is 
the sum of the maximum and minimum task durations. Indeed this may not even be 
the case, e.g., if the two longest duration tasks are close to each other in duration and 
the shortest duration task takes much less time than the second shortest task. As a 
concrete example, if the task durations are 1,8,9,10, the optimum delay is 8 + 9 = 17, 
not 1 + 10. 

In summary, we sort the set of task durations, and pair the shortest, second shortest, 
third shortest, etc. tasks with the longest, second longest, third longest, etc. tasks. For 
example, if the durations are 5,2,1,6,4,4, then on sorting we get 1,2,4,4,5,6, and the 
pairings are (1,6), (2,5), and (4,4). 

private static class PairedTasks { 
public Integer taskl; 
public Integer task2; 

public PairedTasks(Integer taskl, Integer task2) { 
this.taskl = taskl; 
this.task2 = task2; 

> 

} 

public static List<PairedTasks> optimumTaskAssignment( 

Listdnteger> taskDurations) { 

Collections.sort(taskDurations); 

List<PairedTasks> optimumAssignments = new ArrayList<>(); 
for (int i = ®, j = taskDurations.size() - 1; i < j; ++i, --j) { 
optimumAssignments.add( 

new PairedTasks(taskDurations.get(i), taskDurations.get(j))); 

} 

return optimumAssignments; 

} 


The time complexity is dominated by the time to sort, i.e., 0(n log n). 


18.2 Schedule to minimize waiting time 

A database has to respond to a set of client SQL queries. The service time required for 
each query is known in advance. For this application, the queries must be processed 
by the database one at a time, but can be done in any order. The time a query waits 
before its turn comes is called its waiting time. 
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Given service times for a set of queries, compute a schedule for processing the queries 
that minimizes the total waiting time. Return the minimum waiting time. For 
example, if the service times are (2,5,1,3), if we schedule in the given order, the total 
waiting time is 0+(2)+ (2+5)+ (2+5 + 1) = 17. If however, we schedule queries in order 
of decreasing service times, the total waiting time is 0 + (5) + (5 + 3) + (5 + 3 + 2) = 23. 
As we will see, for this example, the minimum waiting time is 10. 

Hint: Focus on extreme values. 

Solution: We can solve this problem by enumerating all schedules and picking the 
best one. The complexity is very high— 0(n\) to be precise, where n is the number of 
queries. 

Intuitively, it makes sense to serve the short queries first. The justification is as 
follows. Since the service time of each query adds to the waiting time of all queries 
remaining to be processed, if we put a slow query before a fast one, by swapping the 
two we improve the waiting time for all queries between the slow and fast query, and 
do not change the waiting time for the other queries. We do increase the waiting time 
of the slow query, but this cancels out with the decrease in the waiting time of the fast 
query. Hence, we should sort the queries by their service time and then process them 
in the order of nondecreasing service time. 

For the given example, the best schedule processes queries in increasing order of 
service times. It has a total waiting time of 0 + (1) + (1 + 2) + (1 + 2 + 3) = 10. Note that 
scheduling queries with longer service times, which we gave as an earlier example, 
is the worst approach. 

public static int minimumTotalWaitingTime (Listdnteger> serviceTimes) { 

// Sort the service times in increasing order. 

Collections.sort(serviceTimes); 

int totalWaitingTime = ®; 

for (int i = ®; i < serviceTimes. size () ; ++i) { 

int numRemainingQueries = serviceTimes.size() - (i + 1); 
totalWaitingTime += serviceTimes.get(i) * numRemainingQueries; 

} 

return totalWaitingTime; 

} 


The time complexity is dominated by the time to sort, i.e., 0(n log n). 


18.3 The interval covering problem 

Consider a foreman responsible for a number of tasks on the factory floor. Each task 
starts at a fixed time and ends at a fixed time. The foreman wants to visit the floor to 
check on the tasks. Your job is to help him minimize the number of visits he makes. 
In each visit, he can check on all the tasks taking place at the time of the visit. A visit 
takes place at a fixed time, and he can only check on tasks taking place at exactly that 
time. For example, if there are tasks at times [0,3], [2,6], [3,4], [6,9], then visit times 
0,2,3,6 cover all tasks. A smaller set of visit times that also cover all tasks is 3,6. In 
the abstract, you are to solve the following problem. 
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You are given a set of closed intervals. Design an efficient algorithm for finding a 
minimum sized set of numbers that covers all the intervals. 

Hint: Think about extremal points. 

Solution: Note that we can restrict our attention to numbers which are endpoints 
without losing optimality. A brute-force approach might be to enumerate every 
possible subset of endpoints, checking for each one if it covers all intervals, and if so, 
determining if it has a smaller size than any previous such subset. Since a set of size 
k has 2 k subsets, the time complexity is very high. 

We could simply return the left end point of each interval, which is fast to compute 
but, as the example in the problem statement showed, may not be the minimum 
number of visit times. Similarly, always greedily picking an endpoint which covers 
the most intervals may also lead to suboptimum results, e.g., consider the six intervals 
[1,2], [2,3], [3,4], [2,3], [3,4], [4,5]. The point 3 appears in four intervals, more than 
any other point. However, if we choose 3, we do not cover [1,2] and [4,5], so we need 
two additional points to cover all six intervals. If we pick 2 and 4, each individually 
covers just three intervals, but combined they cover all six. 

It is a good idea to focus on extreme cases. In particular, consider the interval that 
ends first, i.e., the interval whose right endpoint is minimum. To cover it, we must 
pick a number that appears in it. Furthermore, we should pick its right endpoint, 
since any other intervals covered by a number in the interval will continue to be 
covered if we pick the right endpoint. (Otherwise the interval we began with cannot 
be the interval that ends first.) Once we choose that endpoint, we can remove all 
other covered intervals, and continue with the remaining set. 

The above observation leads to the following algorithm. Sort all the intervals, 
comparing on right endpoints. Select the first interval's right endpoint. Iterate 
through the intervals, looking for the first one not covered by this right endpoint. As 
soon as such an interval is found, select its right endpoint and continue the iteration. 

For the given example, [1,2], [2,3], [3,4], [2,3], [3,4], [4,5], after sorting on right 
endpoints we get [1,2], [2,3], [2,3], [3,4], [3,4], [4,5]. The leftmost right endpoint is 2, 
which covers the first three intervals. Therefore, we select 2, and as we iterate, we see 
it covers [1,2], [2,3], [2,3]. When we get to [3,4], we select its right endpoint, i.e., 4. 
This covers [3,4], [3,4], [4,5]. There are no remaining intervals, so {2,4} is a minimum 
set of points covering all intervals. 

public static class Interval { 
public int left, right; 

public Interval (int 1, int r) { 
this. left = 1; 
this. right = r; 

} 

} 

public static List<Integer> findMinimuraVisits(Listdnterval> intervals) { 
if (intervals.isEmpty()) { 

return Collections.EMPTY_LIST; 

} 
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// Sort intervals based on the right endpoints. 

Collections.sort(intervals, new Comparatordnterval>() { 

©Override 

public int compare(Interval i1, Interval i2) { 
return Integer.compare(il.right, i2.right); 

} 

}); 

Listdnteger> visits = new ArrayList<>() ; 

Integer lastVisitTime = intervals.get(©).right; 
visits.add(lastVisitTime); 
for (Interval interval : intervals) { 
if (interval.left > lastVisitTime) { 

// The current right endpoint, lastVisitTime, will not cover any more 
// intervals. 

lastVisitTime = interval.right; 
visits.add(lastVisitTime) ; 

} 

} 

return visits; 


Since we spend (9(1) time per index, the time complexity after the initial sort is 0(n), 
where n is the number of intervals Therefore, the time taken is dominated by the 
initial sort, i.e., 0(n log n). 



Figure 18.1: An instance of the minimum ray covering problem, with 12 partially overlapping arcs. Arcs 
have been drawn at different distances for illustration. For this instance, six cameras are sufficient, 
corresponding to the six rays. 


Variant: You are responsible for the security of a castle. The castle has a circular 
perimeter. A total of n robots patrol the perimeter—each robot is responsible for a 
closed connected subset of the perimeter, i.e., an arc. (The arcs for different robots 
may overlap.) You want to monitor the robots by installing cameras at the center 
of the castle that look out to the perimeter. Each camera can look along a ray. To 
save cost, you would like to minimize the number of cameras. See Figure 18.1 for an 
example. 

Variant: There are a number of points in the plane that you want to observe. You are 
located at the point (0,0). You can rotate about this point, and your field-of-view is 
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a fixed angle. Which direction should you face to maximize the number of visible 
points? 

Invariants 

An invariant is a condition that is true during execution of a program. Invariants can 
be used to design algorithms as well as reason about their correctness. For example, 
binary search, maintains the invariant that the space of candidate solutions contains 
all possible solutions as the algorithm executes. 

Sorting algorithms nicely illustrate algorithm design using invariants. For ex¬ 
ample, intuitively, selection sort is based on finding the smallest element, the next 
smallest element, etc. and moving them to their right place. More precisely, we work 
with successively larger subarrays beginning at index 0, and preserve the invariant 
that these subarrays are sorted, their elements are less than or equal to the remaining 
elements, and the entire array remains a permutation of the original array. 

As a more sophisticated example, consider Solution 15.7 on Page 267, specifically 
the 0(k) algorithm for generating the first k numbers of the form a + b V2. The key 
idea there is to process these numbers in sorted order. The queues in that code 
maintain multiple invariants: queues are sorted, duplicates are never present, and 
the separation between elements is bounded. 

Invariants boot camp 

Suppose you were asked to write a program that takes as input a sorted array and a 
given value and determines if there are two entries in the array that add up to that 
value. For example, if the array is (-2,1,2,4,7,11), then there are entries adding to 6 
and to 10, but not to 0 and 13. 

There are several ways to solve this problem: iterate over all pairs, or for each 
array entry search for the given value minus that entry. The most efficient approach 
uses invariants: maintain a subarray that is guaranteed to hold a solution, if it exists. 
This subarray is initialized to the entire array, and iteratively shrunk from one side 
or the other. The shrinking makes use of the sortedness of the array. Specifically, if 
the sum of the leftmost and the rightmost elements is less than the target, then the 
leftmost element can never be combined with some element to obtain the target. A 
similar observation holds for the rightmost element. 

public static boolean hasTwoSum(List<Integer> A, int t) { 
int i = ®, j = A.sizeO - 1; 
while (i <= j) { 

if (A.get(i) + A.get(j) == t) { 

return true; 

} else if (A.get ( i) + A.get(j) < t) { 

++i ; 

} else { // A[i] + A[j] > t. 

--j; 

} 

} 

return false; 

} 
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The time complexity is 0(n), where n is the length of the array. The space complexity 
is 0(1), since the subarray can be represented by two variables. 


The key strategy to determine whether to use an invariant when designing an al¬ 
gorithm is to work on small examples to hypothesize the invariant. [Problems 18.4 
and 18.6] 

Often, the invariant is a subset of the set of input space, e.g,. a subarray. [Prob¬ 
lems 18.4 and 18.7] 


18.4 The 3-sum problem 

Design an algorithm that takes as input an array and a number, and determines if there 
are three entries in the array (not necessarily distinct) which add up to the specified 
number. For example, if the array is (11,2,5,7,3) then there are three entries in the 
array which add up to 21 (3,7,11 and 5,5,11). (Note that we can use 5 twice, since 
the problem statement said we can use the same entry more than once.) However, no 
three entries add up to 22. 

Hint: How would you check if a given array entry can be added to two more entries to get the 
specified number? 

Solution: The brute-force algorithm is to consider all possible triples, e.g., by three 
nested for-loops iterating over all entries. The time complexity is 0(n 3 ), where n is 
the length of the array, and the space complexity is 0(1). 

Let A be the input array and t the specified number. We can improve the time 
complexity to 0(n 2 ) by storing the array entries in a hash table first. Then we iterate 
over pairs of entries, and for each A[i] + A[j] we look for t - (A[i] + A[j]) in the hash 
table. The space complexity now is 0(n). 

We can avoid the additional space complexity by first sorting the input. Specifi¬ 
cally, sort A and for each A [i], search for indices j and k such that A [j] + A [k] = t - A [i ]. 
We can do each such search in 0(n log n) time by iterating over A[j] values and doing 
binary search for A[k], 

We can improve the time complexity to 0(n) by starting with A[0] + A[n - 1]. If 
this equals t - A[i\ , we're done. Otherwise, ii A[0] + A[n - 1] < t - A[i], we move to 
A[l] + A[n - 1]—there is no chance of 7l[0] pairing with any other entry to get t - A[i] 
(since A[n- 1] is the largest value in A). Similarly, if A[0] +A[n- 1] > t — A[i], we move 
to A[0] + A[n - 2]. This approach eliminates an entry in each iteration, and spends 
0(1) time in each iteration, yielding an 0(n) time bound to find A[j] and A[k] such 
that A[j] + A[k\ = t - A[i\, if such entries exist. The invariant is that if two elements 
which sum to the desired value exist, they must lie within the subarray currently 
under consideration. 

For the given example, after sorting the array is (2,3,5,7,11). For entry A[0] = 2, 
to see if there are A[j] and A[k] such that j4[0] + A[j] + A[k] = 21, we search for two 
entries that add up to 21 - 2 = 19. 

The code for this approach is shown below. 

public static boolean hasThreeSum(List<Integer> A, int t) { 
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Collections.sort(A); 
for (Integer a : A) { 

// Finds if the sum of two numbers in A equals to t - a. 
if (TwoSum.hasTwoSum(A, t - a)) { 

return true; 

} 

} 

return false; 


The additional space needed is 0( 1), and the time complexity is the sum of the time 
taken to sort, 0(n log n), and then to run the 0(n) algorithm to find a pair in a sorted 
array that sums to a specified value, which is 0(n 2 ) overall. 

Variant: Solve the same problem when the three elements must be distinct. For 
example, if A = (5,2,3,4,3) and t - 9, then A[ 2] + ^4[2] + A[ 2] is not acceptable, 
A[ 2] +A[ 2] + A[ 4] is not acceptable, but A[ 1] + A[2] + A[ 3] and A[ 1] + A[ 3] + A[ 4] are 
acceptable. 

Variant: Solve the same problem when k, the number of elements to sum, is an 
additional input. 

Variant: Write a program that takes as input an array of integers A and an integer 
T, and returns a 3-tuple (A\p],A[q],A[r]) where p,q,r are all distinct, minimizing 
|T - (A[p] + A[q\ + A[r])\, and A[p] < A[r] < A[s]. 

Variant: Write a program that takes as input an array of integers A and an integer 
T, and returns the number of 3-tuples (j p f q,r ) such that A[p] + A[q] + A[r] < T and 
A[p] < A[q] < A[r]. 


18.5 Find the majority element 

Several applications require identification of elements in a sequence which occur 
more than a specified fraction of the total number of elements in the sequence. For 
example, we may want to identify the users using excessive network bandwidth or 
IP addresses originating the most Flypertext Transfer Protocol (HTTP) requests. Here 
we consider a simplified version of this problem. 

You are reading a sequence of strings. You know a priori that more than half the 
strings are repetitions of a single string (the "majority element") but the positions 
where the majority element occurs are unknown. Write a program that makes a 
single pass over the sequence and identifies the majority element. For example, if the 
input is ( b , a, c, a, a, b, a, a, c, a), then a is the majority element (it appears in 6 out of the 
10 places). 

Hint: Take advantage of the existence of a majority element to perform elimination. 

Solution: The brute-force approach is to use a hash table to record the repetition 
count for each distinct element. The time complexity is 0(n), where n is the number 
of elements in the input, but the space complexity is also 0(n). 
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Randomized sampling can be used to identify a majority element with high prob¬ 
ability using less storage, but is not exact. 

The intuition for a better algorithm is as follows. We can group entries into two 
subgroups—those containing the majority element, and those that do not hold the 
majority element. Since the first subgroup is given to be larger in size than the second, 
if we see two entries that are different, at most one can be the majority element. By 
discarding both, the difference in size of the first subgroup and second subgroup 
remains the same, so the majority of the remaining entries remains unchanged. 

The algorithm then is as follows. We have a candidate for the majority element, 
and track its count. It is initialized to the first entry. We iterate through remaining 
entries. Each time we see an entry equal to the candidate, we increment the count. If 
the entry is different, we decrement the count. If the count becomes zero, we set the 
next entry to be the candidate. 

Here is a mathematical justification of the approach. Let's say the majority element 
occurred m times out of n entries. By the definition of majority element, f > \. At 
most one of the two distinct entries that are discarded can be the majority element. 
Hence, after discarding them, the ratio of the number of remaining majority elements 
to the total number of remaining elements is either ^f^y (neither discarded element 
was the majority element) or (one discarded element was the majority element). 
It is simple to verify that if f > then both > \ aR d ^r^y > \- 

For the given example, (b f a,c,a,a,b,a,a,c,a), we initialize the candidate to b. The 
next element, a is different from the candidate, so the candidate's count goes to 0. 
Therefore, we pick the next element c to be the candidate, and its count is 1. The next 
element, a , is different so the count goes back to 0. The next element is a, which is 
the new candidate. The subsequent b decrements the count to 0. Therefore the next 
element, a, is the new candidate, and it has a nonzero count till the end. 

public static String majoritySearch(Iterator<String> sequence) { 

String candidate = 

String iter; 
int candidateCount = ®; 
while (sequence.hasNext()) { 
iter = sequence.next(); 
if (candidateCount == ®) { 
candidate = iter; 
candidateCount = 1; 

} else if (candidate.equals(iter)) { 

++candidateCount; 

} else { 

--candidateCount ; 

} 

} 

return candidate; 

} 


Since we spend 0(1) time per entry, the time complexity is 0(n). The additional space 
complexity is 0(1). 

The code above assumes a majority word exists in the sequence. If no word has a 
strict majority, it still returns a word from the stream, albeit without any meaningful 
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guarantees on how common that word is. We could check with a second pass whether 
the returned word was a majority. Similar ideas can be used to identify words that 
appear more than n/k times in the sequence, as discussed in Problem 25.37 on Page 505. 


18.6 The gasup problem 

In the gasup problem, a number of cities are arranged on a circular road. You need 
to visit all the cities and come back to the starting city. A certain amount of gas is 
available at each city. The amount of gas summed up over all cities is equal to the 
amount of gas required to go around the road once. Your gas tank has unlimited 
capacity. Call a city ample if you can begin at that city with an empty tank, refill at it, 
then travel through all the remaining cities, refilling at each, and return to the ample 
city, without running out of gas at any point. See Figure 18.2 for an example. 



Figure 18.2: The length of the circular road is 3000 miles, and your vehicle gets 20 miles per gallon. 
The distance noted at each city is how far it is from the next city. For this configuration, we can begin 
with no gas at City D, and complete the circuit successfully, so D is an ample city. 


Given an instance of the gasup problem, how would you efficiently compute an ample 
city? You can assume that there exists an ample city. 

Hint: Think about starting with more than enough gas to complete the circuit without gassing 
up. Track the amount of gas as you perform the circuit, gassing up at each city. 

Solution: The brute-force approach is to simulate the traversal from each city. This 
approach has time 0(n 2 ) time complexity, where n is the number of cities. 

Greedy approaches, e.g., finding the city with the most gas, the city closest to 
the next city, or the city with best distance-to-gas ratio, do not work. For the given 
example, A has the most gas, but you cannot get to C starting from A. The city G 
is closest to the next city (A), and has the lowest distance-to-gas ratio (100/10) but it 
cannot get to D. 

We can gain insight by looking at the graph of the amount of gas as we perform 
the traversal. See Figure 18.3 on the following page for an example. The amount of 
gas in the tank could become negative, but we ignore the physical impossibility of 
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that for now. These graphs are the same up to a translation about the Y-axis and a 
cyclic shift about the X-axis. 

In particular, consider a city where the amount of gas in the tank is minimum when 
we enter that city. Observe that it does not depend where we begin from—because 
graphs are the same up to translation and shifting, a city that is minimum for one 
graph will be a minimum city for all graphs. Let z be a city where the amount of gas 
in the tank before we refuel at that city is minimum. Now suppose we pick z as the 
starting point, with the gas present at z. Since we never have less gas than we started 
with at z, and when we return to z we have 0 gas (since it's given that the total amount 
of gas is just enough to complete the traversal) it means we can complete the journey 
without running out of gas. Note that the reasoning given above demonstrates that 
there always exists an ample city. 



(a) Gas vs. distance, starting at A. 



(b) Gas vs. distance, starting at D. 


Figure 18.3: Gas as a function of distance for different starting cities for the configuration in Figure 18.2 
on the preceding page. 


The computation to determine z can be easily performed with a single pass over 
all the cities simulating the changes to amount of gas as we advance. 

private static class CityAndRemainingGas { 
public Integer city; 
public Integer remainingGallons; 

public CityAndRemainingGas(Integer city, Integer remainingGallons) { 
this. city = city; 

this .remainingGallons = remainingGallons; 

} 

} 

private static final int MPG = 2Q; 

// gallons[i] is the amount of gas in city i, and distances[i] is the distance 
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// city i to the next city. 

public static int findAmpleCity(Listdnteger> gallons, 

Listdnteger> distances) { 

int remainingGallons = ®; 

CityAndRemainingGas min = new CityAndRemainingGas(®, ®) ; 
final int numCities = gallons . size() ; 
for (int i = 1; i < numCities; ++i) { 

remainingGallons += gallons . get(i - 1) - distances.get(i - 1) / MPG; 
if (remainingGallons < min.remainingGallons) { 

min = new CityAndRemainingGas(i, remainingGallons); 

} 

} 

return min.city; 


The time complexity is 0{ri), and the space complexity is (9(1). 

Variant: Solve the same problem when you cannot assume that there exists an ample 
city. 


18.7 Compute the maximum water trapped by a pair of vertical lines 

An array of integers naturally defines a set of lines parallel to the Y-axis, starting from 
x = 0 as illustrated in Figure 18.4(a). The goal of this problem is to find the pair of 
lines that together with the X-axis "trap" the most water. See Figure 18.4(b) for an 
example. 



(a) A graphical depiction of the array (1, 2 , 1, 3 , 4 , 4 , 5,6,2,1, 3 , 1, 3 , 2 , 1, 2 , 4 , 1). 


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 

(b) The shaded area between 4 and 16 is the maximum water that can be trapped by the array in (a). 

Figure 18.4: Example of maximum water trapped by a pair of vertical lines. 
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Write a program which takes as input an integer array and returns the pair of entries 
that trap the maximum amount of water. 

Hint: Start with 0 and n- 1 and work your way in. 

Solution: Let A be the array, and n its length. There is a straightforward 0(n 3 ) 
brute-force solution—for each pair of indices (/, j), i < j, the water trapped by the 
corresponding lines is (j - i) X min (A[i],A[j\), which can easily be computed in 0(n) 
time. The maximum of all these is the desired quantity. This algorithm can easily be 
improved to 0(n 2 ) time by keeping the running minimum as we iterate over j. 

In an attempt to achieve a better time complexity, we can try divide-and-conquer. 
We find the maximum water that can be trapped by the left half of A, the right half 
of A, and across the center of A. Finding the maximum water trapped by an element 
on the left half and the right half entails considering combinations of the n/2 entries 
from left half and n/2 entries on the right half. Therefore, the time complexity T(n) of 
this divide-and-conquer approach satisfies T(n ) = 2T(n/2) + 0(n 2 / 4) which solves to 
T(n) = 0(n 2 ). This is no better than the brute-force solution, and considerably trickier 
to code. 

A good starting point is to consider the widest pair, i.e., 0 and n -1. We record the 
corresponding amount of trapped water, i.e., ((n -1) - 0) X min(A [0], A [n -1 ]). Suppose 
A[0] < A[n - 1], Then for any k, the water trapped between 0 and k is less than the 
water trapped between 0 and n- 1, so we focus our attention on the maximum water 
that can be trapped between 1 and n — 1. The converse is true if A[0] > A[n - 1]—we 
need never consider n — 1 again. If A[0] = A[n - 1], we can eliminate both 0 and n - 1 
from further consideration. We use this approach iteratively to continuously reduce 
the subarray that must be explored, while recording the most water trapped so far. 
In essence, we are exploring the best way in which to trade-off width for height. 

For the given example, we begin with (0,17), which has a capacity of 1 x 17 = 17. 
Since the left and right lines have the same height, namely 1, we can advance both, so 
now we consider (1,16). The capacity is 2 X 15 = 30. Since 2 < 4, we move to (2,16). 
The capacity is 1X14 = 14. Since 1 < 4, we move to (3,16). The capacity is 3 X13 = 39. 
Since 3 < 4, we move to (4,16). The capacity is 4 X 12 = 48. Future iterations, which 
we do not show, do not surpass 48, which is the result. 

public static int getMaxTrappedWater(List<Integer> heights) { 
int i = ©, j = heights . size () - 1, maxWater = ©; 
while (i < j) { 

int width = j - i; 

maxWater = Math.max(maxWater, 

width * Math.min(heights.get(i), heights.get(j))); 
if (heights.get(i) > heights.get(j)) { 

--j; 

} else if (heights.get(i) < heights.get(j)) { 

++i ; 

} else { // heights.get(i) == heights . get(j ). 

++i ; 

--j; 

} 

} 

return maxWater; 
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} 


We iteratively eliminate one line or two lines at a time, and we spend 0(1) time per 
iteration, so the time complexity is 0(ri). 


18.8 Compute the largest rectangle under the skyline 

You are given a sequence of adjacent buildings. Each has unit width and an integer 
height. These buildings form the skyline of a city. An architect wants to know the 
area of a largest rectangle contained in this skyline. See Figure 18.5 for an example. 



Figure 18.5: A collection of unit-width buildings, and the largest contained rectangle. The text label 
identifying the building is just below and to the right of its upper left-hand corner. The shaded area is the 
largest rectangle under the skyline. Its area is 2 x (11 - 1). Note that the tallest rectangle is from 7 to 9, 
and the widest rectangle is from 0 to 1, but neither of these are the largest rectangle under the skyline. 


Let A be an array representing the heights of adjacent buildings of unit width. Design 
an algorithm to compute the area of the largest rectangle contained in this skyline. 

Hint: How would you efficiently find the largest rectangle which includes the z'th building, and 
has height A[i\? 

Solution: A brute-force approach is to take each (i,j) pair, find the minimum of 
subarray A[i : ;], and multiply that by j—i + 1. This has time complexity <9(n 3 ), where 
n is the length of A. This can be improved to 0(n 2 ) by iterating over i and then j > i 
and tracking the minimum height of buildings from i to /, inclusive. However, there 
is no reasonable way to further refine this algorithm to get the time complexity below 
0(n 2 ). 

Another brute-force solution is to consider for each i the furthest left and right 
we can go without dropping below A[i] in height. In essence, we want to know the 
largest rectangle that is "supported" by Building i, which can be viewed as acting like 
a "pillar" of that rectangle. For the given example, the largest rectangle supported by 
G extends from 1 to 11, and the largest rectangle supported by F extends from 3 to 6. 

We can easily determine the largest rectangle supported by Building i with a single 
forward and a single backward iteration from i. Since i ranges from 0 to n - 1, the 
time complexity of this approach is 0(n 2 ). 

This brute-force solution can be refined to get a much better time complexity. 
Specifically, suppose we iterate over buildings from left to right. When we process 
Building i, we do not know how far to the right the largest rectangle it supports goes. 
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However, we do know that the largest rectangles supported by earlier buildings 
whose height is greater than A[i] cannot extend past i, since Building i "blocks" these 
rectangles. For example, when we get to F in Figure 18.5 on the preceding page, we 
know that the largest rectangles supported by Buildings B, D, and E (whose heights 
are greater than F 's height) cannot extend past 5. 

Looking more carefully at the example, we see that there's no need to consider B 
when examining F, since C has already blocked the largest rectangle supported by 
B. In addition, there's no reason to consider C: since G's height is the same as C's 
height, and C has not been blocked yet, G and C will have the same largest supported 
rectangle. Generalizing, as we advance through the buildings, all we really need is 
to keep track of is buildings that have not been blocked yet. Additionally, we can 
replace existing buildings whose height equals that of the current building with the 
current building. Call these buildings the set of active pillars. 

Initially there are no buildings in the active pillar set. As we iterate from 0 to 12, the 
active pillar sets are [A], {A,B|, [A, C), \A, C,D), {A, C,D,E}, {A, C,F}, {A, G\, [A, G,H}, 
{A, G, J}, {A, G, /}, {A,K}, {L}, and {L,M}. 

Whenever a building is removed from the active pillar set, we know exactly how 
far to the right the largest rectangle that it supports goes to. For example, when we 
reach C we know B's supported rectangle ends at 2, and when we reach F, we know 
that D and E's largest supported rectangles end at 5. 

When we remove a blocked building from the active pillar set, to find how far to 
the left its largest supported rectangle extends we simply look for the closest active 
pillar that has a lower height. For example, when we reach F, the active pillars 
are {A, C,D,E}. We remove E and D from the set, since both are taller than F. The 
largest rectangle supported by E has height 6 and begins after D, i.e., at 4; its area is 
6 X (5 - 4) = 6. The largest rectangle supported by D has height 5 and begins after C, 
i.e., at 3; its area is 5 X (5 - 3) = 10. 

There are a number of data structures that can be used to store the active pillars 
and the right data structure is key to making the above algorithm efficient. When we 
process a new building we need to find the buildings in the active pillar set that are 
blocked. Because insertion and deletion into the active pillar set take place in last-in 
first-out order, a stack is a reasonable choice for maintaining the set. Specifically, the 
rightmost building in the active pillar set appears at the top. Using a stack also makes 
it easy to find how far to the left the largest rectangle that's supported by a building 
in the active pillar set extends—we simply look at the building below it in the stack. 
For example, when we process F, the stack is A, C, D, E, with E at the top. Comparing 
F's height with E, we see E is blocked so the largest rectangle under E ends at where 
F begins, i.e., at 5. Since the next building in the stack is D, we know that the largest 
rectangle under E begins where D ends, i.e., at 4. 

The algorithm described above is almost complete. The missing piece is what to 
do when we get to the end of the iteration. The stack will not be empty when we 
reach the end—at the very least it will contain the last building. We deal with these 
elements just as we did before, the only difference being that the largest rectangle 
supported by each building in the stack ends at n, where n is the number of elements 
in the array. For the given example, the stack contains L and M when we get to the 
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end, and the largest supported rectangles for both of these end at 13. 


public static int calculateLargestRectangle(Listdnteger> heights) { 

Dequednteger> pillarlndices = new LinkedList<>() ; 

int maxRectangleArea = ®; 

for (int i = ®; i <= heights.size(); ++i) { 

if (!pillarlndices.isEmpty() && i < heights.size() 

&& heights.get(i).equals(heights.get(pillarIndices.peekFirst()))) { 

// Replace earlier building with same height by current building. This 
// ensures the later buildings have the correct left endpoint. 
pillarlndices.removeFirst(); 
pillarlndices.addFirst(i); 

} 

// By iterating to heights.size() instead of heights.size() - 1, we can 
// uniformly handle the computation for rectangle area here. 
while (!pillarlndices.isEmpty() 

&& isNewPillarOrReachEnd(heights, i, pillarlndices.peekFirst())) { 
int height = heights.get(pillarlndices.removeFirst()); 
int width 

= pillarlndices.isEmpty() ? i : i - pillarlndices.peekFirst() - 1; 
maxRectangleArea = Math.max(maxRectangleArea, height * width); 

} 

pillarlndices.addFirst(i); 

} 

return maxRectangleArea; 

} 

private static boolean isNewPillarOrReachEnd(List<Integer> heights, 

int currldx, int lastPillarldx) { 

return currldx < heights.size() 

? heights.get(currldx) < heights.get(lastPillarldx) 

: true; 

} 


The time complexity is 0(n). When advancing through buildings, the time spent 
for building is proportional to the number of pushes and pops performed when 
processing it. Although for some buildings, we may perform multiple pops, in total 
we perform at most n pushes and at most n pops. This is because in the advancing 
phase, an entry i is added at most once to the stack and cannot be popped more 
than once. The time complexity of processing remaining stack elements after the 
advancing is complete is also 0(n) since there are at most n elements in the stack, and 
the time to process each one is 0(1). Thus, the overall time complexity is 0(n). The 
space complexity is 0(n), which is the largest the stack can grow to, e.g., if buildings 
appear in ascending order. 

Variant: Find the largest square under the skyline. 
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Chapter 


Graphs 

Concerning these bridges, it was asked whether anyone could arrange a 
route in such a way that he would cross each bridge once and only once. 

— "The solution of a problem relating to the geometry of position," 

L. Euler, 1741 


Informally, a graph is a set of vertices and connected by edges. Formally, a directed 
graph is a set V of vertices and a set E C V X V of edges. Given an edge e = (u, v), 
the vertex u is its source , and v is its sink . Graphs are often decorated, e.g., by adding 
lengths to edges, weights to vertices, a start vertex, etc. A directed graph can be 
depicted pictorially as in Figure 19.1. 

A path in a directed graph from u to vertex v is a sequence of vertices (v 0 , v \,..., 
where z>o = ti, v n -\ = v, and each 1 ) is an edge. The sequence may consist of a 
single vertex. The length of the path {v 0 , v\, ..., is n - 1. Intuitively, the length of 
a path is the number of edges it traverses. If there exists a path from u to v, v is said 
to be reachable from u. For example, the sequence ( a,c,e,d,h) is a path in the graph 
represented in Figure 19.1. 



A directed acyclic graph (DAG) is a directed graph in which there are no cycles , i.e., 
paths which contain one or more edges and which begin and end at the same vertex. 
See Figure 19.2 on the next page for an example of a directed acyclic graph. Vertices 
in a DAG which have no incoming edges are referred to as sources; vertices which 
have no outgoing edges are referred to as sinks. A topological ordering of the vertices 
in a DAG is an ordering of the vertices in which each edge is from a vertex earlier 
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in the ordering to a vertex later in the ordering. Solution 19.8 on Page 369 uses the 
notion of topological ordering. 



Figure 19.2: A directed acyclic graph. Vertices a,g,m are sources and vertices l,f,h,n are sinks. The 
ordering {a, b, c, e, d, g, h, k, i, /, f, l, m, n) is a topological ordering of the vertices. 


An undirected graph is also a tuple (V, £); however, E is a set of unordered pairs 
of vertices. Graphically, this is captured by drawing arrowless connections between 
vertices, as in Figure 19.3. 



Figure 19.3: An undirected graph. 


If G is an undirected graph, vertices u and v are said to be connected if G contains a 
path from u to v; otherwise, u and v are said to be disconnected. A graph is said to be 
connected if every pair of vertices in the graph is connected. A connected component 
is a maximal set of vertices C such that each pair of vertices in C is connected in G. 
Every vertex belongs to exactly one connected component. 

For example, the graph in Figure 19.3 is connected, and it has a single connected 
component. If edge (h,i) is removed, it remains connected. If additionally ( f,i ) is 
removed, it becomes disconnected and there are two connected components. 

A directed graph is called weakly connected if replacing all of its directed edges with 
undirected edges produces an undirected graph that is connected. It is connected if it 
contains a directed path from u to v or a directed path from v to u for every pair of 
vertices u and v. It is strongly connected if it contains a directed path from u to v and a 
directed path from v to u for every pair of vertices u and v. 

Graphs naturally arise when modeling geometric problems, such as determining 
connected cities. However, they are more general, and can be used to model many 
kinds of relationships. 
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A graph can be implemented in two ways—using adjacency lists or an adjacency 
matrix. In the adjacency list representation, each vertex v, has a list of vertices to which 
it has an edge. The adjacency matrix representation uses a |V| X \ V\ Boolean-valued 
matrix indexed by vertices, with a 1 indicating the presence of an edge. The time and 
space complexities of a graph algorithm are usually expressed as a function of the 
number of vertices and edges. 

A tree (sometimes called a free tree) is a special sort of graph—it is an undirected 
graph that is connected but has no cycles. (Many equivalent definitions exist, e.g., 
a graph is a free tree if and only if there exists a unique path between every pair 
of vertices.) There are a number of variants on the basic idea of a tree. A rooted 
tree is one where a designated vertex is called the root, which leads to a parent-child 
relationship on the nodes. An ordered tree is a rooted tree in which each vertex has 
an ordering on its children. Binary trees, which are the subject of Chapter 10, differ 
from ordered trees since a node may have only one child in a binary tree, but that 
node may be a left or a right child, whereas in an ordered tree no analogous notion 
exists for a node with a single child. Specifically, in a binary tree, there is position as 
well as order associated with the children of nodes. 

As an example, the graph in Figure 19.4 is a tree. Note that its edge set is a subset 
of the edge set of the undirected graph in Figure 19.3 on the preceding page. Given a 
graph G = (V, E), if the graph G' = ( V, E') where E' C E, is a tree, then G' is referred to 
as a spanning tree of G. 



Graphs boot camp 

Graphs are ideal for modeling and analyzing relationships between pairs of objects. 
For example, suppose you were given a list of the outcomes of matches between pairs 
of teams, with each outcome being a win or loss. A natural question is as follows: 
given teams A and B, is there a sequence of teams starting with A and ending with B 
such that each team in the sequence has beaten the next team in the sequence? 

A slick way of solving this problem is to model the problem using a graph. Teams 
are vertices, and an edge from one team to another indicates that the team corre¬ 
sponding to the source vertex has beaten the team corresponding to the destination 
vertex. Now we can apply graph reachability to perform the check. Both DFS and 
BFS are reasonable approaches—the program below uses DFS. 

public static class MatchResult { 
public String winningTeam; 
public String losingTeam; 
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public MatchResult(String winningTeam, String losingTeam) { 
this.winningTeam = winningTeam; 
this.losingTeam = losingTeam; 

} 

} 

public static boolean canTeamABeatTeamB(List<MatchResult> matches, 

String teamA, String teamB) { 

Set<String> visited = new HashSet<>(); 

return isReachableDFS(buildGraph(matches), teamA, teamB, visited); 


private static Map<String, Set<String>> buildGraph( 
List<MatchResult> matches) { 

Map<String, Set<String>> graph = new HashMap<>(); 
for (MatchResult match : matches) { 

Set<String> edges = graph.get(match.winningTeam); 
if (edges == null) { 

edges = new HashSet <>(); 

graph.put(match.winningTeam, edges); 

} 

edges.add(match.losingTeam); 

} 

return graph; 


private static boolean isReachableDFS(Map<String, Set<String>> graph, 

String curr, String dest, 
Set<String> visited) { 

if (curr.equals(dest)) { 

return true; 

} else if (visited.contains(curr) || graph.get(curr) == null) { 
return false; 

> 

visited.add(curr); 

for (String team : graph.get(curr)) { 

if (isReachableDFS(graph, team, dest, visited)) { 
return true; 

} 

} 

return false; 


The time complexity and space complexity are both 0(E), where E is the number of 
outcomes. 

Graph search 

Computing vertices which are reachable from other vertices is a fundamental opera¬ 
tion which can be performed in one of two idiomatic ways, namely depth-first search 
(DFS) and breadth-first search (BFS). Both have linear time complexity— 0(\V\ + \E\) 
to be precise. In the worst-case there is a path from the initial vertex covering all 
vertices without any repeats, and the DFS edges selected correspond to this path, so 
the space complexity of DFS is <9(| V|) (this space is implicitly allocated on the function 
call stack). The space complexity of BFS is also 0(\V\), since in a worst-case there is 
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It's natural to use a graph when the problem involves spatially connected objects, 
e.g., road segments between cities. [Problems 19.1 and 19.2] 

More generally, consider using a graph when you need to analyze any binary 
relationship, between objects, such as interlinked webpages, followers in a social 
graph, etc. [Problems 19.7 and 19.8] 

Some graph problems entail analyzing structure, e.g., looking for cycles or con¬ 
nected components. DFS works particularly well for these applications. [Prob¬ 
lem 19.4] 

Some graph problems are related to optimization, e.g., find the shortest path from 
one vertex to another. BFS, Dijkstra's shortest path algorithm, and minimum 
spanning tree are examples of graph algorithms appropriate for optimization 
problems. [Problem 19.9] 


an edge from the initial vertex to all remaining vertices, implying that they will all be 
in the BFS queue simultaneously at some point. 

DFS and BFS differ from each other in terms of the additional information they 
provide, e.g., BFS can be used to compute distances from the start vertex and DFS can 
be used to check for the presence of cycles. Key notions in DFS include the concept 
of discovery time and finishing time for vertices. 

19.1 Search a maze 

It is natural to apply graph models and algorithms to spatial problems. Consider a 
black and white digitized image of a maze—white pixels represent open areas and 
black spaces are walls. There are two special white pixels: one is designated the 
entrance and the other is the exit. The goal in this problem is to find a way of getting 
from the entrance to the exit, as illustrated in Figure 19.5 on the next page. 

Given a 2D array of black and white entries representing a maze with designated 
entrance and exit points, find a path from the entrance to the exit, if one exists. 

Hint: Model the maze as a graph. 

Solution: A brute-force approach would be to enumerate every possible path from 
entry to exit. However, we know from Solution 17.3 on Page 312 that the number of 
paths is astronomical. Of course, pruning helps, since we can stop as soon as a path 
hits a black pixel, but the worse-case behavior of enumerative approaches is still very 
bad. 

Another approach is to perform a random walk moving from a white pixel to a 
random adjacent white pixel. Given enough time this will find a path, if one exists. 
However, it repeats visits, which retards the progress. The random walk does suggest 
the right way—we should keep track of pixels that we have already visited. This is 
exactly what DFS and BFS do to ensure progress. 

This suggests modeling the maze as a graph. Each vertex corresponds to a white 
pixel. We will index the vertices based on the coordinates of the corresponding pixel, 
i.e., vertex zv h j corresponds to the white entry at (/, j) in the 2D array. Edges model 
adjacent white pixels. 
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(a) A maze. (b) A path from entrance to exit. 
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(c) A shortest path from entrance to 
exit. 


Figure 19.5: An instance of the maze search problem, with two solutions, where S and E denote the 
entrance and exit, respectively. 


Now, run a DFS starting from the vertex corresponding to the entrance. If at 
some point, we discover the exit vertex in the DFS, then there exists a path from the 
entrance to the exit. If we implement recursive DFS then the path would consist of 
all the vertices in the call stack corresponding to previous recursive calls to the DFS 
routine. 

This problem can also be solved using BFS from the entrance vertex on the same 
graph model. The BFS tree has the property that the computed path will be a shortest 
path from the entrance. However BFS is more difficult to implement than DFS since 
in DFS, the compiler implicitly handles the DFS stack, whereas in BFS, the queue has 
to be explicitly coded. Since the problem did not call for a shortest path, it is better to 
use DFS. 

public static class Coordinate { 

public int x, y; 

public Coordinate (int x, int y) { 
this.x = x; 
this.y = y; 

} 

©Override 

public boolean equals(Object o) { 
if (this == o) { 
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return true; 


} 

if (o == null || getClassO != o. getClass ()) { 

return false; 

} 

Coordinate that = (Coordinated; 
if (x != that.x || y != that.y) { 

return false; 

} 

return true; 

} 

©Override 

public int hashCode() { 

return Objects.hash(x, y); 

} 


public static enum Color { WHITE, BLACK } 

public static List<Coordinate> searchMaze(List<List<Color>> maze, 

Coordinate s, Coordinate e) { 
List<Coordinate> path = new ArrayList<>(); 
maze.get(s.x) . set(s .y, Color.BLACK); 
path.add(s); 

if O searchMazeHelper(maze, s, e, path)) { 
path.remove(path.size() - 1); 

} 

return path; // Empty path means no path between s and e. 

} 

// Performs DFS to find a feasible path. 

private static boolean searchMazeHelper(List<List<Color>> maze, 

Coordinate cur, Coordinate e, 
ListcCoordinate> path) { 

if (cur.equals(e)) { 
return true; 

} 

final int [][] SHIFT = {{©, 1}, {1, ®} , {®, -1}, {-1, ®}}; 
for (int [] s : SHIFT) { 

Coordinate next = new Coordinate(cur.x + s[®], cur.y + s[1]); 
if (isFeasible(next, maze)) { 

maze.get(next.x).set(next.y, Color.BLACK); 
path.add(next); 

if (searchMazeHelper(maze, next, e, path)) { 

return true; 

} 

path.remove(path.size() - 1); 

} 

} 

return false; 
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// Checks cur is within maze and is a white pixel. 

private static boolean isFeasible(Coordinate cur, List<List<Color>> maze) { 
return cur.x >= Q &<& cur.x < maze, size () && cur.y >= Q 
&& cur.y < maze.get(cur.x).size() 

&& maze.get(cur.x).get(cur.y) == Color.WHITE; 

} 


The time complexity is the same as that for DFS, namely 0(\V\ + |E|). 


19.2 Paint a Boolean matrix 

Let A be a Boolean 2D array encoding a black-and-white image. The entry A (a, b) can 
be viewed as encoding the color at entry (a, b). Call two entries adjacent if one is to 
the left, right, above or below the other. Note that the definition implies that an entry 
can be adjacent to at most four other entries, and that adjacency is symmetric, i.e., if 
eO is adjacent to entry el, then el is adjacent to eO. 

Define a path from entry eO to entry el to be a sequence of adjacent entries, starting 
at eO , ending at el, with successive entries being adjacent. Define the region associated 
with a point (i, j) to be all points (/', j') such that there exists a path from (i, j) to (/', j') 
in which all entries are the same color. In particular this implies (i, j) and ( i', j') must 
be the same color. 
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Figure 19.6: The color of all squares associated with the first square marked with a x in (a) have been 
recolored to yield the coloring in (b). The same process yields the coloring in (c). 


Implement a routine that takes an n X m Boolean array A together with an entry (x, y) 
and flips the color of the region associated with ( x, y). See Figure 19.6 for an example 
of flipping. 

Hint: Solve this conceptually, then think about implementation optimizations. 

Solution: As with Solution 19.1 on Page 354, graph search can overcome the complex¬ 
ity of enumerative and random search solutions. Specifically, entries can be viewed 
as vertices, with vertices corresponding to adjacent entries begin connected by edges. 

For the current problem, we are searching for all vertices whose color is the same 
as that of (x, y) that are reachable from {x, y). Breadth-first search is natural when 
starting with a set of vertices. Specifically, we can use a queue to store such vertices. 
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The queue is initialized to (x, y). The queue is popped iteratively. Call the popped 
point p. First, we record p's initial color, and then flip its color. Next we examine p 
neighbors. Any neighbor which is the same color as p's initial color is added to the 
queue. The computation ends when the queue is empty. Correctness follows from 
the fact that any point that is added to the queue is reachable from ( x , y) via a path 
consisting of points of the same color, and all points reachable from (x, y) via points 
of the same color will eventually be added to the queue. 

private static class Coordinate { 
public Integer x; 
public Integer y; 

public Coordinate(Integer x, Integer y) { 
this.x = x; 
this.y = y; 

} 

} 

public static void flipColor(List<List<Boolean» A, int x, int y) { 
final int[][] DIRS = {{9, 1}, (9, -1}, {1, 9}, {-1, ®}>; 
boolean color = A.get(x).get(y); 

QueuecCoordinate> q = new LinkedList<>(); 

A.get(x).set(y , !A.get(x).get (y)) ; //Flips. 

q.add(new Coordinate(x, y)) ; 
while (!q.isEmpty()) { 

Coordinate curr = q.element(); 
for (int[] dir : DIRS) { 

Coordinate next = new Coordinate(curr.x + dir[9], curr.y + dir[l]); 
if (next.x >= ® && next.x < A.size() && next.y >= ® 

&& next.y < A.get(next.x).size () 

&& A.get(next.x).get(next.y) == color) { 

// Flips the color. 

A.get(next.x).set(next.y, !color); 
q.add(next); 

} 

} 

q.remove(); 

} 

} 


The time complexity is the same as that of BFS, i.e., 0(mn). The space complexity is a 
little better than the worst-case for BFS, since there are at most 0(m + n) vertices that 
are at the same distance from a given entry. 

We also provide a recursive solution which is in the spirit of DFS. It does not need 
a queue but implicitly uses a stack, namely the function call stack. 

public static void flipColor(ListcList<Boolean>> A, int x, int y) { 
final int [][] DIRS = {{®, 1}, {®, -1}, {1, 9}, {-1, ®}}; 
boolean color = A.get(x).get(y); 

A.get(x).set(y, !color) ; // Flips. 

for (int[] dir : DIRS) { 

int nextX = x + dir[Q], nextY = y + dir[l]; 
if (nextX >= ® && nextX < A.sizeQ <&& nextY >= ® 
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<&& nextY < A.get(nextX).size() && A.get(nextX).get(nextY) == 
flipColor(A, nextX, nextY); 


color) { 


The time complexity is the same as that of DFS. 

Both the algorithms given above differ slightly from traditional BFS and DFS 
algorithms. The reason is that we have a color field already available, and hence do 
not need the auxiliary color field traditionally associated with vertices BFS and DFS. 
Furthermore, since we are simply determining reachability, we only need two colors, 
whereas BFS and DFS traditionally use three colors to track state. (The use of an 
additional color makes it possible, for example, to answer questions about cycles in 
directed graphs, but that is not relevant here.) 

Variant: Design an algorithm for computing the black region that contains the most 
points. 

Variant: Design an algorithm that takes a point (a, b), sets A (a, b) to black, and returns 
the size of the black region that contains the most points. Assume this algorithm 
will be called multiple times, and you want to keep the aggregate run time as low as 
possible. 


19.3 Compute enclosed regions 

This problem is concerned with computing regions within a 2D grid that are enclosed. 
See Figure 19.7 for an illustration of the problem. 



Figure 19.7: Three of the four white squares in (a) are enclosed, i.e., there is no path from any of 
them to the boundary that only passes through white squares, (b) shows the white squares that are not 
enclosed. 


The computational problem can be formalized using 2D arrays of Bs (blacks) and 
Ws (whites). Figure 19.7(a) is encoded by 

B B B B~ 

W B W B 

B W W B ’ 

B B B B 
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Figure 19.7(b) on the previous page is encoded by 

B B B B 
W B B B 
B B B B * 

B B B B 

Let A be a 2D array whose entries are either W or B. Write a program that takes A, 
and replaces all Ws that cannot reach the boundary with a B. 

Hint: It is easier to compute the complement of the desired result. 

Solution: It is easier to focus on the inverse problem, namely identifying Ws that can 
reach the boundary. The reason that the inverse is simpler is that if a W is adjacent 
to a W that is can reach the boundary, then the first W can reach it too. The Ws on 
the boundary are the initial set. Subsequently, we find Ws neighboring the boundary 
Ws, and iteratively grow the set. Whenever we find a new W that can reach the 
boundary, we need to record it, and at some stage search for new Ws from it. A queue 
is a reasonable data structure to track Ws to be processed. The approach amounts to 
breadth-first search starting with a set of vertices rather than a single vertex. 

public static void fillSurroundedRegions(List<List<Character>> board) { 
if (board.isEmpty()) { 
return; 

} 

List<List<Boolean>> visited = new ArrayList<>(board.size()); 
for (int i = ®; i < board. size () ; ++i) { 
visited.add( 

new ArrayList(Collections.nCopies(board.get(i).sizeQ, false))); 

} 

// Identifies the regions that are reachable via white path starting from 

// the first or last columns. 

for (int i = ®; i < board. size () ; ++i) { 

if (board . get (i) . get (®) == ’W’ <&<& !visited.get(i).get (®)) { 
markBoundaryRegion(i, ®, board, visited); 

} 

if (board . get ( i) . get (board . get (i) . size () - 1) == ’W’ 

<&& ! visited . get (i) . get (board . get (i) . size () - 1)) { 
markBoundaryRegion(i, board.get(i).size() - 1, board, visited); 

} 

} 

// Identifies the regions that are reachable via white path starting from 
// the first or last rows. 

for (int j = ®; j < board.get(®).size(); ++j) { 

if (board.get(®).get(j) == ’W’ && !visited.get(®).get(j)) { 
markBoundaryRegion(®, j, board, visited); 

} 

if (board.get(board.size() - l).get(j) == ’W’ 

<&& ! visited . get (board . size () - l).get(j)) { 
markBoundaryRegion(board.size() - 1, j, board, visited); 

} 

} 
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// Marks the surrounded white regions as black. 
for (int i = 1; i < board.size() - 1; ++i) { 

for (int j = 1; j < board. get (i) . size () - 1; ++j) { 

if (board.get(i) . get(j) == ’W’ && !visited.get(i).get(j)) { 
board.get(i).set(j, ’B’); 

} 

} 

} 

} 

private static class Coordinate { 
public Integer x; 
public Integer y; 

public Coordinate(Integer x, Integer y) { 
this.x = x; 
this.y = y; 

} 

} 

private static void markBoundaryRegion(int i, int j, 

List<List<Character>> board, 

List<List<Boolean>> visited) { 
Queue<Coordinate> q = new LinkedList<>(); 
q.add(new Coordinate(i, j)); 
visited.get(i).set(j, true); 

// Uses BFS to traverse this region. 
while (!q.isEmpty()) { 

Coordinate curr = q.poll(); 

final int DIRS[][] = {{®, 1}, {®, -1}, {1, ®>, {-1, ®}}; 
for (int[] dir : DIRS) { 

Coordinate next = new Coordinate(curr.x + dir[®], curr.y + dir[l]); 
if (next.x >= ® && next.x < board.size() <&& next.y >= ® 

&& next.y < board.get(next.x).size() 

&& board.get(next.x).get(next.y) == ’W’ 

&& !visited.get(next.x).get(next.y)) { 
visited.get(next.x).set(next.y, true); 
q.add(next); 

} 

} 

} 


The time and space complexity are the same as those for BFS, namely 0(mn) f where 
m and n are the number of rows and columns in A. 


19.4 Deadlock detection 

High performance database systems use multiple processes and resource locking. 
These systems may not provide mechanisms to avoid or prevent deadlock: a situation 
in which two or more competing actions are each waiting for the other to finish, 
which precludes all these actions from progressing. Such systems must support a 
mechanism to detect deadlocks, as well as an algorithm for recovering from them. 
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One deadlock detection algorithm makes use of a "wait-for " graph to track which 
other processes a process is currently blocking on. In a wait-for graph, processes 
are represented as nodes, and an edge from process P to O implies Q is holding a 
resource that P needs and thus P is waiting for Q to release its lock on that resource. A 
cycle in this graph implies the possibility of a deadlock. This motivates the following 
problem. 

Write a program that takes as input a directed graph and checks if the graph contains 
a cycle. 

Hint: Focus on "back" edges. 

Solution: We can check for the existence of a cycle in G by running DFS on G. Recall 
DFS maintains a color for each vertex. Initially, all vertices are white. When a vertex 
is first discovered, it is colored gray. When DFS finishes processing a vertex, that 
vertex is colored black. 

As soon as we discover an edge from a gray vertex back to a gray vertex, a cycle 
exists in G and we can stop. Conversely, if there exists a cycle, once we first reach 
vertex in the cycle (call it v), we will visit its predecessor in the cycle (call it u) before 
finishing processing v, i.e., we will find an edge from a gray to a gray vertex. In 
summary, a cycle exists if and only if DFS discovers an edge from a gray vertex to a 
gray vertex. Since the graph may not be strongly connected, we must examine each 
vertex, and run DFS from it if it has not already been explored. 

public static class GraphVertex { 

public static enum Color { WHITE, GRAY, BLACK } 

public Color color; 

public List<GraphVertex> edges; 

} 

public static boolean isDeadlocked(List<GraphVertex> G) { 
for (GraphVertex vertex : G) { 

if (vertex.color == GraphVertex.Color.WHITE && hasCycle(vertex)) { 

return true; 

} 

} 

return false; 

} 

private static boolean hasCycle(GraphVertex cur) { 

// Visiting a gray vertex means a cycle. 
if (cur.color == GraphVertex.Color.GRAY) { 

return true; 

} 

cur.color = GraphVertex.Color.GRAY; // Marks current vertex as a gray one. 

// Traverse the neighbor vertices. 
for (GraphVertex next : cur.edges) { 

if (next.color != GraphVertex.Color.BLACK) { 
if (hasCycle(next)) { 
return true; 

} 
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} 

} 

cur.color = GraphVertex.Color.BLACK; // Marks current vertex as black. 

return false; 


The time complexity of DFS is 0(\V\ + |E|): we iterate over all vertices, and spend 
a constant amount of time per edge. The space complexity is <9(|V|), which is the 
maximum stack depth—if we go deeper than |V| calls, some vertex must repeat, 
implying a cycle in the graph, which leads to early termination. 

Variant: Solve the same problem for an undirected graph. 

Variant: Write a program that takes as input an undirected graph, which you can 
assume to be connected, and checks if the graph remains connected if any one edge 
is removed. 


19.5 Clone a graph 

Consider a vertex type for a directed graph in which there are two fields: an integer 
label and a list of references to other vertices. Design an algorithm that takes a 
reference to a vertex u, and creates a copy of the graph on the vertices reachable from 
u. Return the copy of u. 

Hint: Maintain a map from vertices in the original graph to their counterparts in the clone. 

Solution: We traverse the graph starting from u. Each time we encounter a vertex 
or an edge that is not yet in the clone, we add it to the clone. We recognize new 
vertices by maintaining a hash table mapping vertices in the original graph to their 
counterparts in the new graph. Any standard graph traversal algorithm works—the 
code below uses breadth first search. 

public static class GraphVertex { 
public int label; 
public List<GraphVertex> edges; 

public GraphVertex (int label) { 
this.label = label; 

edges = new ArrayList<>(); 

} 

} 

public static GraphVertex cloneGraph(GraphVertex g) { 
if (g == null) { 
return null; 

} 

Map<GraphVertex, GraphVertex> vertexMap = new HashMap<>(); 

Queue<GraphVertex> q = new LinkedList<>(); 
q.add(g); 

vertexMap.put(g, new GraphVertex(g. label)) ; 
while (!q.isEmpty()) { 

GraphVertex v = q.remove(); 
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for (GraphVertex e : v.edges) { 

// Try to copy vertex e. 

if (!vertexMap.containsKey(e)) { 

vertexMap.put(e, new GraphVertex(e. label)) ; 
q.add(e); 

} 

// Copy edge . 

vertexMap.get(v).edges.add(vertexMap.get(e)); 

} 

} 

return vertexMap . get (g) ; 

} 


The space complexity is 0(\V\ + |E|), which is the space taken by the result. Excluding 
the space for the result, the space complexity is <9(|V1)—this comes from the hash 
table, as well as the BFS queue. 


19.6 Making wired connections 

Consider a collection of electrical pins on a printed circuit board (PCB). For each pair 
of pins, there may or may not be a wire joining them. This is shown in Figure 19.8, 
where vertices correspond to pins, and edges indicate the presence of a wire between 
pins. (The significance of the colors is explained later.) 



Figure 19.8: A set of pins and wires between them. 


Design an algorithm that takes a set of pins and a set of wires connecting pairs of 
pins, and determines if it is possible to place some pins on the left half of a PCB, and 
the remainder on the right half, such that each wire is between left and right halves. 
Return such a division, if one exists. For example, the light vertices and dark vertices 
in Figure 19.8 are such division. 

Hint: Model as a graph and think about the implication of an odd length cycle. 

Solution: A brute-force approach might be to try all partitions of the pins into two 
sets. However, the number of such partitions is very high. 

A better approach is to use connectivity information to guide the partitioning. 
Assume the pins are numbered from 0 to p - 1. Create an undirected graph G whose 
vertices are the pins. Add an edge between pairs of vertices if the corresponding pins 
are connected by a wire. For simplicity, assume G is connected; if not, the connected 
components can be analyzed independently. 

Run BFS on G beginning with any vertex v 0 . Assign v 0 arbitrarily to lie on the left 
half. All vertices at an odd distance from vq are assigned to the right half. 
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When performing BFS on an undirected graph, all newly discovered edges will 
either be from vertices which are at a distance d from v 0 to undiscovered vertices 
(which will then be at a distance d +1 from vq) or from vertices which are at a distance 
d to vertices which are also at a distance d. First, assume we never encounter an edge 
from a distance k vertex to a distance k vertex. In this case, each wire is from a distance 
k vertex to a distance k + 1 vertex, so all wires are between the left and right halves. 

If any edge is from a distance k vertex to a distance k vertex, we stop—the pins 
cannot be partitioned into left and right halves as desired. The reason is as follows. 
Let u and v be such vertices. Consider the first common ancestor a in the BFS tree of 
u and v (such an ancestor must exist since the search started at v 0 ). The paths p u and 
p v in the BFS tree from a to u and v are of equal length; therefore, the cycle formed by 
going from a to u via p u , then through the edge ( u , v), and then back to a from v via 
p v has an odd length. A cycle in which the vertices can be partitioned into two sets 
must have an even number of edges—it has to go back and forth between the sets and 
terminate at the starting vertex, and each back and forth adds two edges. Therefore, 
the vertices in an odd length cycle cannot be partitioned into two sets such that all 
edges are between the sets. 

public static class GraphVertex { 
public int d = -1; 

public List<GraphVertex> edges = new ArrayList<>(); 

} 

public static boolean isAnyPlacementFeasible(List<GraphVertex> G) { 
for (GraphVertex v : G) { 

if (v.d == -1) { // Unvisited vertex. 
v.d = 0; 
if (!BFS(v)) { 
return false; 

} 

} 

} 

return true; 

} 

private static boolean BFS(GraphVertex s) { 

Queue<GraphVertex> q = new LinkedList<>(); 
q.add(s); 

while (!q.isEmpty()) { 

for (GraphVertex t : q.peek().edges) { 
if (t.d == -1) { // Unvisited vertex. 
t.d = q.peek().d + 1; 
q.add(t); 

} else if (t.d == q.peekQ.d) { 

return false; 

} 

} 

q .remove(); 

} 

return true; 
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The complexity is the same as for BFS, i.e., 0(p + w) time complexity, where w is the 
number of wires, and 0(p) space complexity. 

Graphs that can be partitioned as described above are known as bipartite graphs. 
Another term for such graphs is 2-colorable (since the vertices can be assigned one of 
two colors without neighboring vertices having the same color). 


19.7 Transform one string to another 

Let s and t be strings and D a dictionary, i.e., a set of strings. Define s to produce t if 
there exists a sequence of strings from the dictionary P = (so, S\, ..., s„_i) such that the 
first string is s, the last string is t, and adjacent strings have the same length and differ 
in exactly one character. The sequence P is called a production sequence. For example, 
if the dictionary is {bat, cot, dog, dag, dot, cat}, then (cat, cot, dot, dog) is production 
sequence. 

Given a dictionary D and two strings s and t, write a program to determine if s 
produces t. Assume that all characters are lowercase alphabets. If s does produce t, 
output the length of a shortest production sequence; otherwise, output -1. 

Hint: Treat strings as vertices in an undirected graph, with an edge between u and v if and only 
if the corresponding strings differ in one character. 

Solution: A brute-force approach may be to explore all strings that differ in one 
character from the starting string, then two characters from the starting string, etc. 
The problem with this approach is that it may explore lots of strings that are outside 
the dictionary. 

A better approach is to be more focused on dictionary words. In particular, it's 
natural to model this problem using graphs. The vertices correspond to strings from 
the dictionary and the edge (u, v) indicates that the strings corresponding to u and 
v differ in exactly one character. Note that the relation "differs in one character" is 
symmetric, so the graph is undirected. 

For the given example, the vertices would be {bat, cot, dog, dag, dot, cat}, and the 
edges would be {(bat, cat), (cot, dot), (cot, cat), (dog, dag), (dog, dot)}. 

A production sequence is simply a path in G, so what we need is a shortest path 
from s to t in G. Shortest paths in an undirected graph are naturally computed using 
BFS. 


private static class StringWithDistance { 
public String candidatestring; 
public Integer distance; 

public StringWithDistance(String candidatestring, Integer distance) { 
this .candidatestring = candidatestring; 
this .distance = distance; 

} 

} 

// Uses BFS to find the least steps of transformation. 

public static int transformString(Set<String> D, String s, String t) { 
Queue<StringWithDistance> q = new LinkedList<>(); 
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D. remove (s) ; // Marks s as visited by erasing it in D. 
q.add(new StringWithDistance(s, ®)); 

StringWithDistance f; 

while ((f = q.poll()) != null) { 

// Returns if we find a match. 
if (f.candidatestring.equals(t)) { 

return f.distance; // Number of steps reaches t. 

} 

// Tries all possible transformations of f.first. 

String str = f.candidatestring; 

for (int i = ®; i < str.length(); ++i) { 

String strStart = i == ® ? "" : str.substring(®, i); 

String strEnd = i + 1 < str.length() ? str.substring(i + 1) : 
for (int j = ®; j < 26; ++ j ) { // Iterates through ’a’ ~ ’z’. 
String modStr = strStart + (char)C’a’ + j) + strEnd; 
if (D.remove(modStr)) { 

q.add(new StringWithDistance(modStr, f.distance + 1)); 

} 

} 

} 

} 

return -1; // Cannot find a possible transformations. 


The number of vertices is the number d of words in the dictionary. The number 
of edges is, in the worst-case, 0(d 2 ). The time complexity is that of BFS, namely 
0(d + d 2 ) = 0{d 2 ). If the string length n is less than d then the maximum number of 
edges out of a vertex is <9(n), implying an 0(nd) bound. 

Variant: An addition chain exponentiation program for computing x n is a finite 
sequence (x,x h ,x l1 ,... ,x n ) where each element after the first is either the square of 
some previous element or the product of any two previous elements. For example, 
the term x 15 can be computed by the following two addition chain exponentiation 
programs. 

PI = {x, X 2 = (x) 2 , x 4 = (x 2 ) 2 , X 8 = (x 4 ) 2 , x 12 = x 8 x 4 , x 14 = x 12 x 2 , X 15 = x 14 x> 

P2 = <x, x 2 = (x) 2 , x 3 = x 2 x, x 5 = x 3 x 2 , x 10 = (x 5 ) 2 , x 15 = x 10 x 5 > 

It is not obvious, but the second program, P2, is the shortest addition chain exponen¬ 
tiation program for computing x 15 . 

Given a positive integer n, how would you compute a shortest addition chain 
exponentiation program to evaluate x"? 

Advanced graph algorithms 

Up to this point we looked at basic search and combinatorial properties of graphs. 
The algorithms we considered were all linear time complexity and relatively 
straightforward—the major challenge was in modeling the problem appropriately. 
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There are four classes of complex graph problems that can be solved efficiently, 
i.e., in polynomial time. Most other problems on graphs are either variants of these 
or, very likely, not solvable by polynomial time algorithms. These four classes are: 

• Shortest path —given a graph, directed or undirected, with costs on the edges, 
find the minimum cost path from a given vertex to all vertices. Variants include 
computing the shortest paths for all pairs of vertices, and the case where costs 
are all nonnegative. 

• Minimum spanning tree —given an undirected graph G = (V,E), assumed to 
be connected, with weights on each edge, find a subset E' of the edges with 
minimum total weight such that the subgraph G' = (V, E') is connected. 

• Matching —given an undirected graph, find a maximum collection of edges 
subject to the constraint that every vertex is incident to at most one edge. The 
matching problem for bipartite graphs is especially common and the algorithm 
for this problem is much simpler than for the general case. A common variant 
is the maximum weighted matching problem in which edges have weights and 
a maximum weight edge set is sought, subject to the matching constraint. 

• Maximum flow —given a directed graph with a capacity for each edge, find the 
maximum flow from a given source to a given sink, where a flow is a function 
mapping edges to numbers satisfying conservation (flow into a vertex equals 
the flow out of it) and the edge capacities. The minimum cost circulation 
problem generalizes the maximum flow problem by adding lower bounds on 
edge capacities, and for each edge, a cost per unit flow. 

These four problem classes have polynomial time algorithms and can be solved 
efficiently in practice for very large graphs. Algorithms for these problems tend to 
be specialized, and the natural approach does not always work best. For example, it 
is natural to apply divide-and-conquer to compute the MST as follows. Partition the 
vertex set into two subsets, compute MSTs for the subsets independently, and then 
join these two MSTs with an edge of minimum weight between them. Figure 19.9 
shows how this algorithm can lead to suboptimal results. 



(a) A weighted undirected graph, (b) An MST built by divide-and- (c) An optimum MST. 
conquer from the MSTs on {a,b,c) 
and {d,e,f\. The edge ( b,e ) is the 
lightest edge connecting the two 
MSTs. 


Figure 19.9: Divide-and-conquer applied to the MST problem is suboptimum—the MST in (b) has 
weight 18, but the MST in (c) has weight 14. 


In this chapter we restrict our attention to shortest-path problems. 
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19.8 Team photo day— 2 


How would you generalize your solution to Problem 14.8 on Page 248, to determine 
the largest number of teams that can be photographed simultaneously subject to the 
same constraints? 

Hint: Form a DAG in which paths correspond to valid placements. 

Solution: Let G be the DAG with vertices corresponding to the teams as follows and 
edges from vertex X to Y iff Team X can be placed behind Team Y. 

Every sequence of teams where a team can be placed behind its predecessor 
corresponds to a path in G. To find the longest such sequence, we simply need to 
find the longest path in the DAG G. We can do this, for example, by topologically 
ordering the vertices in G; the longest path terminating at vertex v is the maximum 
of the longest paths terminating at v's fan-ins concatenated with v itself. 

public static class GraphVertex { 

public List<GraphVertex> edges = new ArrayList<>(); 
public int maxDistance = 1; 
public boolean visited = false; 

} 

public static int findLargestNumberTeams(List<GraphVertex> G) { 
Deque<GraphVertex> orderedVertices = buildTopologicalOrdering(G); 
return findLongestPath(orderedVertices); 

} 

private static Deque<GraphVertex> buildTopologicalOrdering( 

List<GraphVertex> G) { 

Deque<GraphVertex> orderedVertices = new LinkedList<>(); 
for (GraphVertex g : G) { 
if (!g.visited) { 

DFS(g, orderedVertices); 

} 

} 

return orderedVertices; 

} 

private static int findLongestPath(Deque<GraphVertex> orderedVertices) { 
int maxDistance = ®; 

while (!orderedVertices.isEmpty()) { 

GraphVertex u = orderedVertices.peekFirst(); 
maxDistance = Math.max(maxDistance, u.maxDistance); 
for (GraphVertex v : u.edges) { 

v.maxDistance = Math.max(v.maxDistance, u.maxDistance + 1); 

} 

orderedVertices.removeFirst(); 

} 

return maxDistance; 

} 

private static void DFS(GraphVertex cur, Deque<GraphVertex> orderedVertices) { 
cur.visited = true; 

for (GraphVertex next : cur.edges) { 
if (!next.visited) { 
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DFS(next, orderedVertices) ; 

} 

} 

orderedVertices.addFirst(cur) ; 


The topological ordering computation is 0(\V\ + \E\) and dominates the computation 
time. Clearly |V| is the number of teams. The number of edges E depends on the 
heights, but can be as high as 0(\V\ 2 ), e.g., when there is a path of length \ V\ — 1. 

Variant: Let T = {To, Ti,..., T„_i} be a set of tasks. Each task runs on a single generic 
server. Task T; has a duration t„ and a set P, (possibly empty) of tasks that must be 
completed before T, can be started. The set is feasible if there does not exist a sequence 
of tasks (To, Ti,..., T„_i, To) starting and ending at the same task such that for each 
consecutive pair of tasks, the first task must be completed before the second task can 
begin. 

Given an instance of the task scheduling problem, compute the least amount of 
time in which all the tasks can be performed, assuming an unlimited number of 
servers. Explicitly check that the system is feasible. 


19.9 Compute a shortest path with fewest edges 

In the usual formulation of the shortest path problem, the number of edges in the 
path is not a consideration. For example, considering the shortest path problem from 
a to h in Figure 19.1 on Page 350, the sum of the edge costs on the path (a, c f e, d, h ) is 
22, which is the same as for path Both are shortest paths, but the 

latter has three more edges. 

Heuristically, if we did want to avoid paths with a large number of edges, we can 
add a small amount to the cost of each edge. However, depending on the structure 
of the graph and the edge costs, this may not result in the shortest path. 

Design an algorithm which takes as input a graph G = ( V, E), directed or undirected, 
a nonnegative cost function on E, and vertices s and t; your algorithm should output 
a path with the fewest edges amongst all shortest paths from s to t. 

Hint: Change the edge cost and cast it as an instance of the standard shortest path problem. 

Solution: Dijkstra's shortest path algorithm uses scalar values for edge length. How¬ 
ever, it can easily be modified to the case where the edge weight is a pair if addition 
and comparison can be defined over these pairs. In this case, if the edge cost is c, 
we say the length of the edge is given by the pair (c, 1). We define addition to be 
just component-wise addition. Hence, if we sum up the edge lengths over a path, 
we essentially get the total cost and the number of edges in the path. The compare 
function first compares the total cost, and breaks ties on the number of edges. We 
can run Dijkstra's shortest path algorithm with this compare function and find the 
shortest path that requires the least number of edges. 

Since a heap does not support efficient updates, it is more convenient to use a BST 
than a heap to implement the algorithm. 


370 



private static class VertexWithDistance { 
public GraphVertex vertex; 
public Integer distance; 

public VertexWithDistance(GraphVertex vertex, Integer distance) { 
this .vertex = vertex; 
this.distance = distance; 

} 

} 

private static class DistanceWithFewestEdges { 
public Integer distance; 
public Integer minNumEdges; 

public DistanceWithFewestEdges(Integer distance, Integer minNumEdges) { 
this.distance = distance; 
this.minNumEdges = minNumEdges; 

} 

} 

public static class GraphVertex implements Comparable<GraphVertex> { 
public DistanceWithFewestEdges distanceWithFewestEdges 
= new DistanceWithFewestEdges(Integer.MAX_VALUE, ®); 
public List<VertexWithDistance> edges = new ArrayList<>(); 
public int id; // The id of this vertex. 

public GraphVertex pred = null; // The predecessor in the shortest path. 
©Override 

public int compareTo(GraphVertex o) { 
if (distanceWithFewestEdges.distance 

!= o.distanceWithFewestEdges.distance) { 
return Integer.compare(distanceWithFewestEdges.distance, 

o.distanceWithFewestEdges.distance); 

} 

if (distanceWithFewestEdges.minNumEdges 

!= o.distanceWithFewestEdges.minNumEdges) { 
return Integer.compare(distanceWithFewestEdges.minNumEdges, 

o.distanceWithFewestEdges.minNumEdges); 

} 

return Integer.compare(id, o.id); 

} 

©Override 

public boolean equals(Object obj) { 

if (obj == null || !(obj instanceof GraphVertex)) { 

return false; 

} 

if (this == obj) { 
return true; 

} 

GraphVertex that = (GraphVertex)obj; 
return id == that.id 

&& distanceWithFewestEdges.distance.equals( 

that.distanceWithFewestEdges.distance) 

&& distanceWithFewestEdges.minNumEdges.equals( 

that.distanceWithFewestEdges.minNumEdges); 
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} 


©Override 

public int hashCode() { 

return Obj ects.hash(distanceWithFewestEdges.distance, 

distanceWithFewestEdges.minNumEdges); 


} 


} 


public static void di j kstraShortestPath(GraphVertex s, GraphVertex t) { 
// Initialization of the distance of starting point. 
s.distanceWithFewestEdges = new DistanceWithFewestEdges(®, ®) ; 
SortedSet<GraphVertex> nodeSet = new TreeSet<>(); 
nodeSet.add(s); 


while (!nodeSet.isEmpty()) { 

// Extracts the minimum distance vertex from heap. 
GraphVertex u = nodeSet.first(); 
if (u.equals(t)) { 
break; 

} 

nodeSet.remove(nodeSet.first()); 


// Relax neighboring vertices of u. 
for (VertexWithDistance v : u.edges) { 

int vDistance = u.distanceWithFewestEdges.distance + v.distance; 
int vNumEdges = u.distanceWithFewestEdges.minNumEdges + 1; 
if (v.vertex.distanceWithFewestEdges.distance > vDistance 

|| (v.vertex.distanceWithFewestEdges.distance == vDistance 

&& v.vertex.distanceWithFewestEdges.minNumEdges > vNumEdges)) { 
nodeSet.remove(v.vertex); 
v.vertex.pred = u; 
v.vertex.distanceWithFewestEdges 

= new DistanceWithFewestEdges(vDistance, vNumEdges); 
nodeSet.add(v.vertex); 

} 

} 


// Outputs the shortest path with fewest edges. 
outputShortestPath(t); 


private static void outputShortestPath(GraphVertex v) { 
if (v != null) { 

outputShortestPath(v.pred); 

System.out.print(v.id + " "); 

} 

} 


The time complexity is that of the basic implementation of Dijkstra's algorithm, i.e., 
0((|E| + |V|)log|V|). 

Variant: A flight is specified by a start-time, originating city, destination city, and 
arrival-time (possibly on a later day). A time-table is a set of flights. Given a time¬ 
table, a starting city, a starting time, and a destination city, how would you compute 
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the soonest you could get to the destination city? Assume all flights start and end on 
time, that you need 60 minutes between flights, and a flight departing from A to B 
cannot arrive earlier at B than another flight from AtoB which departed earlier. 
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Chapter 


Parallel Computing 

The activity of a computer must include the proper reacting to a pos¬ 
sibly great variety of messages that can be sent to it at unpredictable 
moments, a situation which occurs in all information systems in 
which a number of computers are coupled to each other. 

— "Cooperating sequential processes 
E. W. Dijkstra, 1965 


Parallel computation has become increasingly common. For example, laptops and 
desktops come with multiple processors which communicate through shared mem¬ 
ory. High-end computation is often done using clusters consisting of individual 
computers communicating through a network. 

Parallelism provides a number of benefits: 

• High performance—more processors working on a task (usually) means it is 
completed faster. 

• Better use of resources—a program can execute while another waits on the disk 
or network. 

• Fairness—letting different users or programs share a machine rather than have 
one program run at a time to completion. 

• Convenience—it is often conceptually more straightforward to do a task using 
a set of concurrent programs for the subtasks rather than have a single program 
manage all the subtasks. 

• Fault tolerance—if a machine fails in a cluster that is serving web pages, the 
others can take over. 

Concrete applications of parallel computing include graphical user interfaces 
(GUI) (a dedicated thread handles UI actions while other threads are, for example, 
busy doing network communication and passing results to the UI thread, result¬ 
ing in increased responsiveness), Java virtual machines (a separate thread handles 
garbage collection which would otherwise lead to blocking, while another thread is 
busy running the user code), web servers (a single logical thread handles a single 
client request), scientific computing (a large matrix multiplication can be split across 
a cluster), and web search (multiple machines crawl, index, and retrieve web pages). 

The two primary models for parallel computation are the shared memory model, in 
which each processor can access any location in memory, and the distributed memory 
model, in which a processor must explicitly send a message to another processor to 
access its memory. The former is more appropriate in the multicore setting and the 
latter is more accurate for a cluster. The questions in this chapter are focused on the 
shared memory model. 
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Writing correct parallel programs is challenging because of the subtle interactions 
between parallel components. One of the key challenges is races—two concurrent 
instruction sequences access the same address in memory and at least one of them 
writes to that address. Other challenges to correctness are 

• starvation (a processor needs a resource but never gets it; e.g.. Problem 20.6 on 
Page 383), 

• deadlock (Thread A acquires Lock LI and Thread B acquires Lock L2, following 
which A tries to acquire L2 and B tries to acquire LI), and 

• livelock (a processor keeps retrying an operation that always fails). 

Bugs caused by these issues are difficult to find using testing. Debugging them is also 
difficult because they may not be reproducible since they are usually load dependent. 
It is also often true that it is not possible to realize the performance implied by 
parallelism—sometimes a critical task cannot be parallelized, making it impossible to 
improve performance, regardless of the number of processors added. Similarly, the 
overhead of communicating intermediate results between processors can exceed the 
performance benefits. 

The problems in this chapter focus on thread-level parallelism. Problems con¬ 
cerned with parallelism on distributed memory architectures, e.g., cluster computing, 
are usually not meant to be coded; they are popular design and architecture prob¬ 
lems. Problems 21.9 on Page 398, 21.10 on Page 399, 21.11 on Page 400, and 21.17 on 
Page 406 cover some aspects of cluster-level parallelism. 

Parallel computing boot camp 

A semaphore is a very powerful synchronization construct. Conceptually, a 
semaphore maintains a set of permits. A thread calling acquire() on a semaphore 
waits, if necessary, until a permit is available, and then takes it. A thread calling re- 
leaseQ on a semaphore adds a permit and notifies threads waiting on that semaphore, 
potentially releasing a blocking acquirer. The program below shows how to im¬ 
plement a semaphore in Java, using synchronization, wait(), and notify() language 
primitives. (The Java concurrency library provides a full-featured implementation of 
semaphores which should be used in practice.) 

public class Semaphore { 

private final int MAX.AVAILABLE; 
private int taken; 

public Semaphore (int maxAvailable) { 
this .MAX_AVAILABLE = maxAvailable; 
this. taken = ®; 

} 

public synchronized void acquire() throws InterruptedException { 
while (this. taken == MAX.AVAILABLE) { 
wait () ; 

} 

this .taken++; 

} 

public synchronized void release() throws InterruptedException { 
this .taken--; 


375 



this .notifyAHQ ; 


} 

} 


Start with an algorithm that locks aggressively and is easily seen to be correct. 
Then add back concurrency, while ensuring the critical parts are locked. [Prob¬ 
lems 20.1 and 20.6] 

When analyzing parallel code, assume a worst-case thread scheduler. In particu¬ 
lar, it may choose to schedule the same thread repeatedly, it may alternate between 
two threads, it may starve a thread, etc. 

Try to work at a higher level of abstraction. In particular, know the concurrency 
libraries—don't implement your own semaphores, thread pools, deferred execu¬ 
tion, etc. (You should know how these features are implemented, and implement 
them if asked to.) [Problem 20.4] 


20.1 Implement caching for a multithreaded dictionary 

The program below is part of an online spell correction service. Clients send as input 
a string, and the service returns an array of strings in its dictionary that are closest 
to the input string (this array could be computed, for example, using Solution 17.2 
on Page 309). The service caches results to improve performance. Critique the 
implementation and provide a solution that overcomes its limitations. 

public static class UnsafeSpellCheckService extends SpellCheckService { 
private static final int MAX.ENTRIES = 3; 

private static LinkedHashMap<String, String[]> cachedClosestStrings 
= new LinkedHashMap<String, String[]>() { 

protected boolean removeEldestEntry(Map.Entry eldest) { 
return size() > MAX.ENTRIES; 

} 

}; 


public static void service(ServiceRequest req, ServiceResponse resp) { 
String w = req.extractWordToCheckFromRequest(); 
if (cachedClosestStrings.containsKey(w)) { 

resp.encodeIntoResponse(cachedClosestStrings.get(w)); 
return; 

} 

String [] closestToLastWord = Spell.closestlnDictionary(w); 
cachedClosestStrings.put(w, closestToLastWord) ; 

} 

} 


Hint: Look for races, and lock as little as possible to avoid reducing throughput. 

Solution: The solution has a race condition. Suppose clients A and B make concurrent 
requests, and the service launches a thread per request. Suppose the thread for 
request A finds that the input string is present in the cache, and then, immediately 
after that check, the thread for request B is scheduled. Suppose this thread's lookup 


376 




fails, so it computes the result, and adds it to the cache. If the cache is full, an entry 
will be evicted, and this may be the result for the string passed in request A. Now 
when request A is scheduled back, it does a lookup for the value corresponding to its 
input string, expecting it to be present (since it checked that that string is a key in the 
cache). However, the cache will return null. 

A thread-safe solution would be to synchronize every call to the service. In this 
case, only one thread could be executing the method and there is no races between 
cache reads and writes. However, it also leads to poor performance—only one thread 
can execute the service call at a time. 

The solution is to lock just the part of the code that operates on the cached values— 
specifically, the check on the cached value and the updates to the cached values: 

In the program below, multiple threads can be concurrently computing closest 
strings. This is good because the calls take a long time (this is why they are cached). 
Locking ensures that the read assignment on a hit and write assignment on completion 
are atomic. 

public static class SafeSpellCheckService extends SpellCheckService { 
private static final int MAX_ENTRIES = 3; 

private static LinkedHashMap<String, String[]> cachedClosestStrings 
= new LinkedHashMap<String, String[]>() { 

protected boolean removeEldestEntry(Map.Entry eldest) { 
return size() > MAX.ENTRIES; 

} 

}; 


public static void service(ServiceRequest req, ServiceResponse resp) { 
String w = req.extractWordToCheckFromRequest(); 
synchronized (S2Alternative. class) { 

if (cachedClosestStrings.containsKey(w)) { 

resp.encodeIntoResponse(cachedClosestStrings.get(w)); 
return; 

} 

} 

String [] closestToLastWord = Spell.closestlnDictionary (w) ; 
synchronized (S2Alternative. class) { 

cachedClosestStrings.put(w, closestToLastWord); 

} 

} 

} 


Variant: Threads It on execute a method called critical(). Before this, they execute a 
method called rendezvous() . The synchronization constraint is that only one thread can 
execute criticalQ at a time, and all threads must have completed executing rendezvous() 
before criticalO can be called. You can assume n is stored in a variable n that is 
accessible from all threads. Design a synchronization mechanism for the threads. All 
threads must execute the same code. Threads may call criticalO multiple times, and 
you should ensure that a thread cannot call criticalO a (k + l)th time until all other 
threads have completed their /cth calls to criticalO. 
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20.2 Analyze two unsynchronized interleaved threads 


Threads tl and tl each increment an integer variable N times, as shown in the code 
below. This program yields nondeterministic results. Usually, it prints 2 N but some¬ 
times it prints a smaller value. The problem is more pronounced for large N. As a 
concrete example, on one run the program output 1320209 when N = 1000000 was 
specified at the command line. 

public static class IncrementThread implements Runnable { 
public void run() { 

for (int i = ®; i < TwoThreadlncrementDriver. N ; i++) { 
TwoThreadlncrementDriver.counter++; 

} 

} 

} 

public static class TwoThreadlncrementDriver { 
public static int counter; 
public static int N; 

public static void main(String[] args) throws Exception { 

N = (args.length > ®) ? new Integer(args[®]) : 1®®; 

Thread tl = new Thread(new IncrementThread()); 

Thread t2 = new Thread(new IncrementThread()); 

11. start () ; 
t2 . start(); 
tl.join(); 
t2.join(); 

System.out.println(counter); 

} 

} 


What are the maximum and minimum values that could be printed by the program 
as a function of N? 

Hint: Be as perverse as you can when scheduling the threads. 

Solution: First, note that the increment code is unguarded, which opens up the 
possibility of its value being determined by the order in which threads that write to 
it are scheduled by the thread scheduler. 

The maximum value is 2N. This occurs when the thread scheduler runs one thread 
to completion, followed by the other thread. 

When N = 1, the minimum value for the count variable is 1: tl reads, tl reads, tl 
increments and writes, then tl increments and writes. When N > 1, the final value 
of the count variable must be at least 2. The reasoning is as follows. There are two 
possibilities. A thread, call it T, performs a read-increment-write-read-increment- 
write without the other thread writing between reads, in which case the written value 
is at least 2. If the other thread now writes a 1, it has not yet completed, so it will 
increment at least once more. Otherwise, T's second read returns a value of 1 or more 
(since the other thread has performed at least one write). 
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N-1 iterations 


t2 


tl 



N-1 iterations 


Figure 20.1: Worst-case schedule for two unsynchronized threads. 


The lower bound of 2 is achieved according to the following thread schedule: 

• tl loads the value of the counter, which is 0. 

• tl executes the loop N-1 times. 

• tl doesn't know that the value of the counter changed and writes 1 to it. 

• tl loads the value of the counter, which is 1. 

• tl executes the loop for the remaining N-1 iterations. 

• tl doesn't know that the value of the counter has changed, and writes 2 to the 
counter. 

This schedule is depicted in Figure 20.1. 


20.3 Implement synchronization for two interleaving threads 

Thread tl prints odd numbers from 1 to 100; Thread tl prints even numbers from 1 
to 100. 

Write code in which the two threads, running concurrently, print the numbers from 1 
to 100 in order. 

Hint: The two threads need to notify each other when they are done. 

Solution: A brute-force solution is to use a lock which is repeatedly captured by 
the threads. A single variable, protected by the lock, indicates who went last. The 
drawback of this approach is that it employs the busy waiting antipattem: processor 
time that could be used to execute a different task is instead wasted on useless activity. 

Below we present a solution based on the same idea, but one that avoids busy 
locking by using Java's wait() and notify() primitives. 

public static class OddEvenMonitor { 

public static final boolean 0DD_TURN = true; 
public static final boolean EVEN_TURN = false; 
private boolean turn = 0DD_TURN; 
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// Need synchronized in order to call wait(), see 
// http://stackoverflow.com/questions/2779484 for discussion 
public synchronized void waitTurn(boolean oldTurn) { 
while (turn != oldTurn) { 
try { 
wait () ; 

} catch (InterruptedException e) { 

System.out.println("InterruptedException in wait(): " + e); 

} 

} 

// Move on, it’s our turn. 

} 

// Need synchronized in order to call notifyC) 
public synchronized void toggleTurnO { 
turn A = true; 
notify () ; 

} 

} 

public static class OddThread extends Thread { 
private final OddEvenMonitor monitor; 

public OddThread(OddEvenMonitor monitor) { this. monitor = monitor; } 
©Override 

public void run() { 

for (int i = 1; i <= 100; i += 2) { 

monitor.waitTurn(OddEvenMonitor.0DD_TURN); 

System.out,println("i = " + i); 
monitor . toggleTurnO ; 

} 

} 

} 

public static class EvenThread extends Thread { 
private final OddEvenMonitor monitor; 

public EvenThread(OddEvenMonitor monitor) { this. monitor = monitor; } 
©Override 

public void run() { 

for (int i = 2; i <= 100; i += 2) { 

monitor.waitTurn(OddEvenMonitor.EVEN_TURN); 

System.out.println("i = " + i); 
monitor . toggleTurnO ; 

} 

} 


public static void main(String[] args) throws InterruptedException { 
OddEvenMonitor monitor = new OddEvenMonitor(); 

Thread tl = new OddThread(monitor); 

Thread t2 = new EvenThread(monitor); 

11.start O ; 
t2.start O ; 
tl.join() ; 
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t2.join() ; 


} 


20.4 Implement a thread pool 

The following program, implements part of a simple HTTP server: 

public static class SingleThreadWebServer { 
public static final int PORT = 888®; 

public static void main(String[] args) throws IOException { 
ServerSocket serversock = new ServerSocket(PORT); 
for (;;) { 

Socket sock = serversock.accept(); 
processReq(sock); 

} 

} 


Suppose you find that the program has poor performance because it frequently blocks 
on I/O. What steps could you take to improve the program's performance? Feel free 
to use any utilities from the standard library, including concurrency classes. 

Hint: Use multithreading, but control the number of threads. 

Solution: The first attempt to solve this problem might be to launch a new thread per 
request rather than process the request itself: 


public static class ConcurrentWebServer { 
private static final int SERVERPORT = 8888; 
public static void main(String[] args) throws IOException { 

final ServerSocket serversocket = new ServerSocket(SERVERPORT); 
while (true) { 

final Socket connection = serversocket.accept(); 

Runnable task = new Runnable() { 

public void run() { Worker.handleRequest(connection); } 

}; 

new Thread(task).start(); 

} 

} 

} 


The problem with this approach is that we do not control the number of threads 
launched. A thread consumes a nontrivial amount of resources, such as the time taken 
to start and end the thread and the memory used by the thread. For a lightly-loaded 
server, this may not be an issue but under load, it can result in exceptions that are 
challenging, if not impossible, to handle. 

The right trade-off is to use a thread pool. As the name implies, this is a collection of 
threads, the size of which is bounded. A thread pool can be implemented relatively 
easily using a blocking queue, i.e., a queue which blocks the writing thread on a put 
until the queue is empty. However, since the problem statement explicitly allows us 
to use library routines, we can use the thread pool implementation provided in the 
Executor framework, which is the approach used below. 


381 



public static class ThreadPoolWebServer { 
private static final int NTHREADS = 188; 
private static final int SERVERPORT = 8888; 

private static final Executor exec = Executors.newFixedThreadPool(NTHREADS); 

public static void main(String[] args) throws IOException { 

ServerSocket serversocket = new ServerSocket(SERVERPORT); 

while (true) { 

final Socket connection = serversocket.accept(); 

Runnable task = new Runnable() { 

public void run() { Worker.handleRequest(connection); } 

}; 

exec.execute(task); 

} 

} 

} 


20.5 Deadlock 

When threads need to acquire multiple locks to enter a critical section, deadlock can 
result. As an example, suppose both T1 and T2 need to acquire locks L and M. If 
T1 first acquires L, and then T2 then acquires M, they end up waiting on each other 
forever. 

Identify a concurrency bug in the program below, and modify the code to resolve the 
issue. 

public static class Account { 
private int balance; 
private int id; 
private static int globalld; 

Account (int balance) { 
this. balance = balance; 
this. id = ++globalId; 

} 

private boolean move(Account to, int amount) { 
synchronized (this) { 
synchronized (to) { 

if (amount > balance) { 
return false; 

} 

to.balance += amount; 
this. balance -= amount; 

System.out.println("returning true"); 
return true; 

} 

} 

} 

public static void transfer (final Account from, final Account to, 

final int amount) { 

Thread transfer = new Thread(new Runnable() { 
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public void run() { from.move(to, amount); } 
}); 

transfer.start() ; 

} 

} 


Solution: Suppose l/l initiates a transfer to 1/2, and immediately afterwards, 1/2 
initiates a transfer to L/l. Since each transfer takes place in a separate thread, it's 
possible for the first thread to lock L/l and then the second thread to be scheduled in 
and take the lock LZ2. The program is now deadlocked—each of the two threads is 
waiting for the lock held by the other thread. 

One solution is to have a global lock which is acquired by the transfer method. 
The drawback is that it blocks transfers that are unrelated, e.g., 1/3 cannot transfer to 
1/4 if there is a pending transfer from 1/5 to t/6. 

The canonical way to avoid deadlock is to have a global ordering on locks and 
acquire them in that order. Since accounts have a unique integer id, the update below 
is all that is needed to solve the deadlock. 


Account lockl = (id < to.id) ? this : to; 

Account lock2 = (id < to.id) ? to : this; 
synchronized (lockl) { 

// Does not matter if lockl equals lock2: since Java locks are 
// reentrant, we will re-acquire lock2. 
synchronized (lock2) { 


20.6 The readers-writers problem 

Consider an object s which is read from and written to by many threads. (For example, 
s could be the cache from Problem 20.1 on Page 376.) You need to ensure that no thread 
may access s for reading or writing while another thread is writing to s. (Two or more 
readers may access s at the same time.) 

One way to achieve this is by protecting s with a mutex that ensures that two 
threads cannot access s at the same time. However, this solution is suboptimal 
because it is possible that a reader Rl has locked s and another reader R2 wants to 
access s. Reader R2 does not have to wait until Rl is done reading; instead, R2 should 
start reading right away. 

This motivates the first readers-writers problem: protect s with the added con¬ 
straint that no reader is to be kept waiting if s is currently opened for reading. 

Implement a synchronization mechanism for the first readers-writers problem. 

Hint: Track the number of readers. 

Solution: We want to keep track of whether the string is being read from, as well as 
whether the string is being written to. Additionally, if the string is being read from, 
we want to know the number of concurrent readers. We achieve this with a pair of 
locks—a read lock and a write lock—and a read counter locked by the read lock. 

A reader proceeds as follows. It locks the read lock, increments the counter, and 
releases the read lock. After it performs its reads, it locks the read lock, decrements the 
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counter, and releases the read lock. A writer locks the write lock, then performs the 
following in an infinite loop. It locks the read lock, checks to see if the read counter 
is 0; if so, it performs its write, releases the read lock, and breaks out of the loop. 
Finally, it releases the write lock. As in Solution 20.3 on Page 379, we use wait-notify 
primitives to avoid busy waiting. 


// LR and LW are static members of type Object in the RW class. 

// They serve as read and write locks. The static integer 
// field readCount in RW tracks the number of readers. 
public static class Reader extends Thread { 
public void run() { 
while (true) { 

synchronized (RW.LR) { RW.readCount++; } 

System.out.printIn(RW.data); 
synchronized (RW.LR) { 

RW. readCount-- ; 

RW.LR.notifyO ; 

} 

Task.doSomeThingElse(); 

} 

} 

} 

public static class Writer extends Thread { 
public void run() { 
while (true) { 

synchronized (RW.LW) { 
boolean done = false; 
while (!done) { 

synchronized (RW.LR) { 
if (RW.readCount == ®) { 

RW.data = new Date().toString(); 
done = true; 

} else { 

// Use wait/notify to avoid busy waiting. 
try { 

// Protect against spurious notify, see 

// stackoverflow.com do-spurious-wakeups- actually-happen . 
while (RW.readCount != ®) { 

RW.LR.wait(); 

} 

} catch (InterruptedException e) { 

System.out.println("InterruptedException in Writer wait"); 

} 

} 

} 

} 

} 

Task.doSomeThingElse(); 

} 

} 

} 
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20.7 The readers-writers problem with write preference 


Suppose we have an object s as in Problem 20.6 on Page 383. In the solution to 
Problem 20.6 on Page 383, a reader R1 may have the lock; if a writer W is waiting for 
the lock and then a reader R2 requests access, R2 will be given priority over W. If this 
happens often enough, W will starve. Instead, suppose we want W to start as soon 
as possible. 

This motivates the second readers-writers problem: protect s with "writer- 
preference", i.e., no writer, once added to the queue, is to be kept waiting longer 
than absolutely necessary. 

Implement a synchronization mechanism for the second readers-writers problem. 
Hint: Force readers to acquire a write lock. 

Solution: We want to give writers the preference. We achieve this by modifying 
Solution 20.6 on Page 383 to have a reader start by locking the write lock and then im¬ 
mediately releasing it. In this way, a writer who acquires the write lock is guaranteed 
to be ahead of the subsequent readers. 

Variant: The specifications to Problems 20.6 on Page 383 and 20.7 allow starvation— 
the first may starve writers, the second may starve readers. The third readers-writers 
problem adds the constraint that neither readers nor writers should starve. Implement 
a synchronization mechanism for the third readers-writers problem. 


20.8 Implement a Timer class 

Consider a web-based calendar in which the server hosting the calendar has to per¬ 
form a task when the next calendar event takes place. (The task could be sending an 
email or a Short Message Service (SMS).) Your job is to design a facility that manages 
the execution of such tasks. 

Develop a timer class that manages the execution of deferred tasks. The timer con¬ 
structor takes as its argument an object which includes a run method and a string¬ 
valued name field. The class must support—(1.) starting a thread, identified by name, 
at a given time in the future; and (2.) canceling a thread, identified by name (the cancel 
request is to be ignored if the thread has already started). 

Hint: There are two aspects—data structure design and concurrency. 

Solution: The two aspects to the design are the data structures and the locking 
mechanism. 

We use two data structures. The first is a min-heap in which we insert key-value 
pairs: the keys are run times and the values are the thread to run at that time. A 
dispatch thread runs these threads; it sleeps from call to call and may be woken up 
if a thread is added to or deleted from the pool. If woken up, it advances or retards 
its remaining sleep time based on the top of the min-heap. On waking up, it looks 
for the thread at the top of the min-heap—if its launch time is the current time, the 
dispatch thread deletes it from the min-heap and executes it. It then sleeps till the 
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launch time for the next thread in the min-heap. (Because of deletions, it may happen 
that the dispatch thread wakes up and finds nothing to do.) 

The second data structure is a hash table with thread ids as keys and entries in the 
min-heap as values. If we need to cancel a thread, we go to the min-heap and delete 
it. Each time a thread is added, we add it to the min-heap; if the insertion is to the 
top of the min-heap, we interrupt the dispatch thread so that it can adjust its wake 
up time. 

Since the min-heap is shared by the update methods and the dispatch thread, we 
need to lock it. The simplest solution is to have a single lock that is used for all read 
and writes into the min-heap and the hash table. 


20.9 Test the Collatz conjecture in parallel 

In Problem 13.13 on Page 230 and its solution we introduced the Collatz conjecture 
and heuristics for checking it. In this problem, you are to build a parallel checker 
for the Collatz conjecture. Specifically, assume your program will run on a multicore 
machine, and threads in your program will be distributed across the cores. Your 
program should check the Collatz conjecture for every integer in [1, U] where U is an 
input to your program. 

Design a multi-threaded program for checking the Collatz conjecture. Make full use 
of the cores available to you. To keep your program from overloading the system, 
you should not have more than n threads running at a time. 

Hint: Use multithreading for performance—take care to minimize threading overhead. 

Solution: Heuristics for pruning checks on individual integers are discussed in So¬ 
lution 13.13 on Page 230. The aim of this problem is implementing a multi-threaded 
checker. We could have a master thread launch n threads, one per number, starting 
with 1,2,..., x. The master thread would keep track of what number needs to be 
processed next, and when a thread returned, it could re-assign it the next unchecked 
number. 

The problem with this approach is that the time spent executing the check in an 
individual thread is very small compared to the overhead of communicating with the 
thread. The natural solution is to have each thread process a subrange of [1, If]. We 
could do this by dividing [1, If] into n equal sized subranges, and having Thread i 
handle the zth subrange. 

The heuristics for checking the Collatz conjecture take longer on some integers 
than others, and in the strategy above there is the potential of a situation arising 
where one thread takes much longer to complete than the others, which leads to most 
of the cores being idle. 

A good compromise is to have threads handle smaller intervals, which are still 
large enough to offset the thread overhead. We can maintain a work-queue consisting 
of unprocessed intervals, and assigning these to returning threads. The Java Executor 
framework is ideally suited to implementing this, and an implementation is given in 
the code below: 


// Performs basic unit of work, i.e., checking CH for an interval 
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public static class MyRunnable implements Runnable { 
public int lower; 
public int upper; 

MyRunnable(int lower, int upper) { 
this.lower = lower; 
this.upper = upper; 

} 

©Override 

public void run() { 

for (int i = lower; i <= upper; ++i) { 

Collatz.CollatzCheck(i, new HashSet<BigInteger>()) ; 

} 

} 


// Checks an individual number 

public static boolean CollatzCheck(BigInteger aNum, Set<BigInteger> visited) { 
if (aNum.equals(Biglnteger.ONE)) { 
return true; 

} else if (visited.contains(aNum)) { 
return false; 

} 

visited.add(aNum); 

if (aNum.getLowestSetBit() == 1) { // Odd number. 
return CollatzCheck( 

new Biglnteger("3").multiply(aNum).add(BigInteger.ONE), visited); 

} else { // Even number. 

return CollatzCheck(aNum.shiftRight(1), visited); // Divide by 2. 

} 

} 

public static boolean CollatzCheck(int aNum, Set<BigInteger> visited) { 
Biglnteger b = new Biglnteger (new Integer(aNum).toString()); 
return CollatzCheck(b, visited); 


public static ExecutorService execute() { 

// Uses the Executor framework for task assignment and load balancing 
List<Thread> threads = new ArrayList<Thread>(); 

ExecutorService executor = Executors.newFixedThreadPool(NTHREADS); 
for (int i = ®; i < (N / RANGESIZE); ++i) { 

Runnable worker = new MyRunnable(i * RANGESIZE +1, (i + 1) * RANGESIZE); 
executor.execute(worker); 

} 

executor.shutdown(); 
return executor; 
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Part III 

Domain Specific Problems 




Design Problems 


■ Don't be fooled by the many books on complexity or by the 

many complex and arcane algorithms you find in this book 
or elsewhere. Although there are no textbooks on simplicity, 
simple systems work and complex don't. 

— "Transaction Processing: Concepts and Techniques," 

J. Gray, 1992 

You may be asked in an interview how to go about creating a set of services or a larger 
system, possibly on top of an algorithm that you have designed. These problems are 
fairly open-ended, and many can be the starting point for a large software project. 

In an interview setting when someone asks such a question, you should have a con¬ 
versation in which you demonstrate an ability to think creatively, understand design 
trade-offs, and attack unfamiliar problems. You should sketch key data structures 
and algorithms, as well as the technology stack (programming language, libraries, 
OS, hardware, and services) that you would use to solve the problem. 

The answers in this chapter are presented in this context—they are meant to be 
examples of good responses in an interview and are not comprehensive state-of-the- 
art solutions. 

We review patterns that are useful for designing systems in Table 21.1. Some 
other things to keep in mind when designing a system are implementation time, 
extensibility, scalability, testability, security, internationalization, and IP issues. 

Table 21.1: System design patterns. 

Design principle Key points 

Algorithms and Data Identify the basic algorithms and data structures 
Structures 

Decomposition Split the functionality, architecture, and code into man¬ 

ageable, reusable components. 

Scalability Break the problem into subproblems that can be solved 

relatively independently on different machines. Shard 
data across machines to make it fit. Decouple the 
servers that handle writes from those that handle reads. 
Use replication across the read servers to gain more 
performance. Consider caching computation and later 
look it up to save work. 
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Decomposition 


Good decompositions are critical to successfully solving system-level design prob¬ 
lems. Functionality, architecture, and code all benefit from decomposition. 

For example, in our solution to designing a system for online advertising (Prob¬ 
lem 21.15 on Page 404), we decompose the goals into categories based on the stake 
holders. We decompose the architecture itself into a front-end and a back-end. The 
front-end is divided into user management, web page design, reporting functional¬ 
ity, etc. The back-end is made up of middleware, storage, database, cron services, 
and algorithms for ranking ads. The design of TpX (Problem 21.6 on Page 395) and 
Connexus (Problem 21.14 on Page 403) also illustrate such decompositions. 

Decomposing code is a hallmark of object-oriented programming. The subject of 
design patterns is concerned with finding good ways to achieve code-reuse. Broadly 
speaking, design patterns are grouped into creational, structural, and behavioral pat¬ 
terns. Many specific patterns are very natural—strategy objects, adapters, builders, 
etc., appear in a number of places in our codebase. Freeman et al.'s "Head First Design 
Patterns" is, in our opinion, the right place to study design patterns. 

Scalability 

In the context of interview questions parallelism is useful when dealing with scale, i.e., 
when the problem is too large to fit on a single machine or would take an unacceptably 
long time on a single machine. The key insight you need to display is that you know 
how to decompose the problem so that 

• each subproblem can be solved relatively independently, and 

• the solution to the original problem can be efficiently constructed from solutions 
to the subproblems. 

Efficiency is typically measured in terms of central processing unit (CPU) time, ran¬ 
dom access memory (RAM), network bandwidth, number of memory and database 
accesses, etc. 

Consider the problem of sorting a petascale integer array. If we know the distri¬ 
bution of the numbers, the best approach would be to define equal-sized ranges of 
integers and send one range to one machine for sorting. The sorted numbers would 
just need to be concatenated in the correct order. If the distribution is not known then 
we can send equal-sized arbitrary subsets to each machine and then merge the sorted 
results, e.g., using a min-heap. Details are given in Solution 21.9 on Page 398. 

The solutions to Problems 21.8 on Page 397 and 21.17 on Page 406 also illustrate 
the use of parallelism. 

Caching is a great tool whenever computations are repeated. For example, the 
central idea behind dynamic programming is caching results from intermediate com¬ 
putations. Caching is also extremely useful when implementing a service that is 
expected to respond to many requests over time, and many requests are repeated. 
Workloads on web services exhibit this property. Solution 20.1 on Page 376 sketches 
the design of an online spell correction service; one of the key issues is performing 
cache updates in the presence of concurrent requests. Solution 20.9 on Page 386 
shows how multithreading combines with caching in code which tests the Collatz 
hypothesis. 
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21.1 Design a spell checker 


Designing a good spelling correction system can be challenging. We discussed 
spelling correction in the context of edit distance (Problem 17.2 on Page 309). How¬ 
ever, in that problem, we only computed the Levenshtein distance between a pair of 
strings. A spell checker must find a set of words that are closest to a given word from 
the entire dictionary. Furthermore, the Levenshtein distance may not be the right 
distance function when performing spelling correction—it does not take into account 
the commonly misspelled words or the proximity of letters on a keyboard. 

How would you build a spelling correction system? 

Hint: Start with an appropriate notion of distance between words. 

Solution: The basic idea behind most spelling correction systems is that the mis¬ 
spelled word's Levenshtein distance from the intended word tends to be very small 
(one or two edits). Hence, if we keep a hash table for all the words in the dictionary 
and look for all the words that have a Levenshtein distance of 2 from the text, it is 
likely that the intended word will be found in this set. If the alphabet has m characters 
and the search text has n characters, we need to perform 0(n 2 m 2 ) hash table lookups. 
More precisely, for a word of length n, we can pick any two characters and change 
them to any other character in the alphabet. The total number of ways of selecting 
any two characters is n(n - l)/2, and each character can be changed to one of (m - 1) 
other chars. Therefore, the number of lookups is n(n - 1 )(m - l) 2 /2. 

The intersection of the set of all strings at a distance of two or less from a word 
and the set of dictionary words may be large. It is important to provide a ranked list 
of suggestions to the users, with the most likely candidates are at the beginning of 
the list. There are several ways to achieve this: 

• Typing errors model—often spelling mistakes are a result of typing errors. 
Typing errors can be modeled based on keyboard layouts. 

• Phonetic modeling—a big class of spelling errors happen when the person 
spelling it knows how the words sounds but does not know the exact spelling. 
In such cases, it helps to map the text to phonemes and then find all the words 
that map to the same phonetic sequence. 

• History of refinements—often users themselves provide a great amount of data 
about the most likely misspellings by first entering a misspelled word and 
then correcting it. This historic data is often immensely valuable for spelling 
correction. 

• Stemming—often the size of a dictionary can be reduced by keeping only the 
stemmed version of each word. (This entails stemming the query text.) 

21.2 Design a solution to the stemming problem 

When a user submits the query "computation" to a search engine, it is quite possible 
he might be interested in documents containing the words "computers", "compute", 
and "computing" also. If you have several keywords in a query, it becomes difficult 
to search for all combinations of all variants of the words in the query. 
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Stemming is the process of reducing all variants of a given word to one common 
root, both in the query string and in the documents. An example of stemming 
would be mapping { computers , computer, compute } to compute. It is almost impossible 
to succinctly capture all possible variants of all words in the English language but a 
few simple rules can get us most cases. 

Design a stemming algorithm that is fast and effective. 

Hint: The examples are suggestive of general rules. 

Solution: Stemming is a large topic. Here we mention some basic ideas related to 
stemming—this is in no way a comprehensive discussion on stemming approaches. 

Most stemming systems are based on simple rewrite rules, e.g., remove suffixes of 
the form "es", "s", and "ation". Suffix removal does not always work. For example, 
wolves should be stemmed to wolf. To cover this case, we may have a rule that 
replaces the suffix "ves" with "f". 

Most rules amount to matching a set of suffixes and applying the corresponding 
transformation to the string. One way of efficiently performing this is to build a finite 
state machine based on all the rules. 

A more sophisticated system might have exceptions to the broad rules based on 
the stem matching some patterns. The Porter stemmer, developed by Martin Porter, 
is considered to be one of the most authoritative stemming algorithms in the English 
language. It defines several rules based on patterns of vowels and consonants. 

Other approaches include the use of stochastic methods to learn rewrite rules and 
n-gram based approaches where we look at the surrounding words to determine the 
correct stemming for a word. 


21.3 Plagiarism detector 

Design an efficient algorithm that takes as input a set of text files and returns pairs of 
files which have substantial commonality. 

Hint: Design a hash function which can incrementally hash S[i :i + k- 1] for i = 0,1,2,_ 

Solution: We will treat each file as a string. We take a pair of files as having substantial 
commonality if they have a substring of length k in common, where k is a measure 
of commonality. (Later, we will have a deeper discussion as to the validity of this 
model.) 

Let /; be the length of the string corresponding to the ith file. For each such string 
we can compute /, - k + 1 hash codes, one for each k length substring. 

We insert these hash codes in a hash table G, recording which file each code 
corresponds to, and the offset the corresponding substring appears at. A collision 
indicates that the two length-/: substrings are potentially the same. 

Since we are computing hash code for each k length substring, it is important 
for efficiency to have a hash function which can be updated incrementally when we 
delete one character and add another. Solution 7.13 on Page 109 describes such a hash 
function. 
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In addition, it is important to have many slots in G, so that collisions of unequal 
strings is rare. A total of Yifhih ~k + 1) strings are added to the hash table. If k is small 
relative to the string length and G has significantly fewer slots than the total number 
of characters in the strings, then we are certain to have collisions. 

If it is not essential to return an exact answer, we can save storage by only consid¬ 
ering a subset of substrings, e.g., those whose hash codes have Os in the last b bits. 
This means that on average we consider d- of the total set of substrings (assuming 
the hash function does a reasonable job of spreading keys). 

The solution presented above can lead to many false positives. For example, if 
each file corresponds to an HTML page, all pages with a common embedded script 
of length k or longer will show up as having substantial overlap. We can account for 
this through preprocessing, e.g., parsing the documents and removing headers. (This 
may require multiple passes with some manual inspection driving the process.) This 
solution will also not work if, for example, the files are similar in that they are the 
exact same program, but with ids renamed. (It is however, resilient to code blocks 
being moved around.) By normalizing ids, we can cast the problem back into one of 
finding common substrings. 

The approach we have just described is especially effective when the number of 
files is very large and the files are spread over many servers. In particular, the map- 
reduce framework can be used to achieve the effect of a single G spread over many 
servers. 


21.4 Pair users by attributes 

You are building a social network where each user specifies a set of attributes. You 
would like to pair each user with another unpaired user that specified the same set 
of attributes. 

You are given a sequence of users where each user has a unique 32-bit integer key 
and a set of attributes specified as strings. When you read a user, you should pair 
that user with another previously read user with identical attributes who is currently 
unpaired, if such a user exists. If the user cannot be paired, you should add him to 
the unpaired set. 

Hint: Map sets of attributes to strings. 

Solution: First, we examine the algorithmic core of this problem. Later we will 
consider implementation details, particularly those related to scale. 

A brute-force algorithm is to compare each new user's attributes with the attributes 
of users in the unpaired set. This leads to 0(n 2 ) time complexity, where n is the number 
of users. 

To improve the time complexity, we need a way to find users who have the same 
attributes as the new user. A hash table whose keys are subsets of attributes and 
values are users is the perfect choice. This leaves us with the problem of designing a 
hash function which is suitable for subsets of attributes. 

If the total number of possible attributes is small, we can represent a subset of 
attributes with a bit array, where each index represents a specific attribute. Specifically, 
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we store a 1 at an index if that attribute is present. We can then use any hash function 
for bit arrays. The time complexity to process the n users is a hash lookup per user, 
i.e., 0{nm), where m is the number of distinct attributes. The space complexity is also 
0(nm)—n entries, each of size m. 

If the set of possible attributes is large, and most users have a small subset of those 
attributes, the bit vector representation for subsets is inefficient from both a time and 
space perspective. A better approach to represent sparse subsets is directly record the 
elements. To ensure equal subsets have equal hash codes, we need to represent the 
subsets in a unique way. One approach is to sort the elements. We can view the sorted 
sequence of attributes as a single string, namely the concatenation the individual ele¬ 
ments. We then use a hash function for strings. For example, if the possible attributes 
are {USA, Senior, Income, Prime Customer}, and a user has attributes {USA, Income}, 
we represent his set of attributes as the string // Income,USA ,/ . 

The time complexity of this approach is <9(M), where M is the sum of the sizes of 
the attribute sets for all users. 

Now we consider implementation issues. Suppose the network is small enough 
that a single machine can store it. For such a system, you can use database with Users 
table. Attributes table, and a join table between them. Then for every attribute you 
will find matches (or no match) and based on criteria you can decide if user could join 
a group or start a new one. If you do not like to use a database and want to reinvent 
the wheel, you still can use similar approach and get a list of matching user IDs for 
every attribute. To do it use reverse indexes, with a string hash for quick lookup in 
storage. Assuming arrays are returned sorted there is an easy way to merge arrays 
for multiple attributes and make a decision about matching to a group / starting new 
group. 

Now suppose the network is too large to fit on a single machine. Because of 
number of users it makes sense to process problem on multiple machines—each will 
store subset of attributes and may return results as user IDs. We will need to make 
two merge operations: 

• Identical searches—when a single attribute lookup was performed on many 
machines. We will need to merge returned sorted IDs into big sorted list of IDs. 

• Searches by different attributes—we will need to merge those attributes if they 
all are present in all lists (or if the match criteria is above X%, e.g. 3/4 are 
matching). 

Since in all likelihood we do not need to perform real-time matching we could use 
the Consumer-Producer pattern: pick up users from a queue and perform a search, 
thereby limit the number of simultaneous requests. 

21.5 Design a system for detecting copyright infringement 

YouTV.com is a successful online video sharing site. Hollywood studios complain that 
much of the material uploaded to the site violates copyright. 

Design a feature that allows a studio to enter a set V of videos that belong to it, and 
to determine which videos in the YouTV.com database match videos in V. 

Hint: Normalize the video format and create signatures. 
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Solution: If we replaced videos everywhere with documents, we could use the tech¬ 
niques in Solution 21.3 on Page 392, where we looked for near duplicate documents 
by computing hash codes for each length-A: substring. 

Videos differ from documents in that the same content may be encoded in many 
different formats, with different resolutions, and levels of compression. 

One way to reduce the duplicate video problem to the duplicate document problem 
is to re-encode all videos to a common format, resolution, and compression level. This 
in itself does not mean that two videos of the same content get reduced to identical 
files—the initial settings affect the resulting videos. However, we can now "signature" 
the normalized video. 

A trivial signature would be to assign a 0 or a 1 to each frame based on whether 
it has more or less brightness than average. A more sophisticated signature would 
be a 3 bit measure of the red, green, and blue intensities for each frame. Even more 
sophisticated signatures can be developed, e.g., by taking into account the regions on 
individual frames. The motivation for better signatures is to reduce the number of 
false matches returned by the system, and thereby reduce the amount of time needed 
to review the matches. 

The solution proposed above is algorithmic. However, there are alternative ap¬ 
proaches that could be effective: letting users flag videos that infringe copyright (and 
possibly rewarding them for their effort), checking for videos that are identical to 
videos that have previously been identified as infringing, looking at meta-information 
in the video header, etc. 

Variant: Design an online music identification service. 

21.6 Design TpX 

The TpX system for typesetting beautiful documents was designed by Don Knuth. Un¬ 
like GUI based document editing programs, TpX relies on a markup language, which 
is compiled into a device independent intermediate representation. TpX formats text, 
lists, tables, and embedded figures; supports a very rich set of fonts and mathematical 
symbols; automates section numbering, cross-referencing, index generation; exports 
an API; and much more. 

How would you implement TpX? 

Hint: There are two aspects—building blocks (e.g., fonts, symbols) and hierarchical layout. 

Solution: Note that the problem does not ask for the design of TpX, which itself 
is a complex problem involving feature selection, and language design. There are 
a number of issues common to implementing any such program: programming 
language selection, lexing and parsing input, error handling, macros, and scripting. 

Two key implementation issues specific to TpX are a specifying fonts and symbols 
(e.g., A, b,/, Yj, §, *±0/ and assembling a document out of components. 

Focusing on the second aspect, a reasonable abstraction is to use a rectangular 
bounding box to describe components. The description is hierarchical: each individ¬ 
ual symbol is a rectangle, lines and paragraphs are made out of these rectangles and 
are themselves rectangles, as are section titles, tables and table entries, and included 
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images. A key algorithmic problem is to assemble these rectangles, while preserving 
hard constraints on layout, and soft constraints on aesthetics. See also Problem 17.11 
on Page 327 for an example of the latter. 

Turning our attention to symbol specification, the obvious approach is to use 
a 2D array of bits to represent each symbol. This is referred to as a bit-mapped 
representation. The problem with bit-mapped fonts is that the resolution required 
to achieve acceptable quality is very high, which leads to huge documents and font- 
libraries. Different sizes of the same symbol need to be individually mapped, as do 
italicized and bold-face versions. 

A better approach is to define symbols using mathematical functions. A reason¬ 
able approach is to use a language that supports quadratic and cubic functions, and 
elementary graphics transformations (rotation, interpolation, and scaling). This ap¬ 
proach overcomes the limitations of bit-mapped fonts—parameters such as aspect 
ratio, font slant, stroke width, serif size, etc. can be programmed. 

Other implementation issues include enabling cross-referencing, automatically 
creating indices, supporting colors, and outputting standard page description formats 
( e -gv PDF)). 

Donald Knuth's book "Digital Typography" describes in great detail the design and 
implementation of TpX. 


21.7 Design a search engine 

Keyword-based search engines maintain a collection of several billion documents. 
One of the key computations performed by a search engine is to retrieve all the 
documents that contain the keywords contained in a query. This is a nontrivial task 
in part because it must be performed in a few tens of milliseconds. 

Here we consider a smaller version of the problem where the collection of docu¬ 
ments can fit within the RAM of a single computer. 

Given a million documents with an average size of 10 kilobytes, design a program 
that can efficiently return the subset of documents containing a given set of words. 

Hint: Build on the idea of a book's index. 

Solution: The predominant way of doing this is to build inverted indices. In an 
inverted index, for each word, we store a sequence of locations where the word 
occurs. The sequence itself could be represented as an array or a linked list. Location 
is defined to be the document ID and the offset in the document. The sequence is 
stored in sorted order of locations (first ordered by document ID, then by offset). 
When we are looking for documents that contain a set of words, what we need to do 
is find the intersection of sequences for each word. Since the sequences are already 
sorted, the intersection can be done in time proportional to the aggregate length of 
the sequences. We list a few optimizations below. 

• Compression —compressing the inverted index helps both with the ability to in¬ 
dex more documents as well as memory locality (fewer cache misses). Since 
we are storing sorted sequences, one way of compressing is to use delta corn- 
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pression where we only store the difference between the successive entries. The 
deltas can be represented in fewer bits. 

• Caching —the distribution of queries is usually fairly skewed and it helps a great 
deal to cache the results of some of the most frequent queries. 

• Frequency-based optimization —since search results often do not need to return 
every document that matches (only top ten or so), only a fraction of highest 
quality documents can be used to answer most of the queries. This means that 
we can make two inverted indices, one with the high quality documents that 
stays in RAM and one with the remaining documents that stays on disk. This 
way if we can keep the number of queries that require the secondary index to a 
small enough number, then we can still maintain a reasonable throughput and 
latency. 

• Intersection order —since the total intersection time depends on the total size of 
sequences, it would make sense to intersect the words with smaller sets first. 
For example, if we are looking for "USA GDP 2009", it would make sense to 
intersect the lists for GDP and 2009 before trying to intersect the sequence for 
USA. 

We could also build a multilevel index to improve accuracy on documents. For a high 
priority web page, we can decompose the page into paragraphs and sentences, which 
are indexed individually. That way the intersections for the words might be within 
the same context. We can pick results with closer index values from these sequences. 
See the sorted array intersection problem 14.1 on Page 236 and digest problem 13.7 
on Page 218 for related issues. 


21.8 Implement PageRank 

The PageRank algorithm assigns a rank to a web page based on the number of 
"important" pages that link to it. The algorithm essentially amounts to the following: 
(1.) Build a matrix A based on the hyperlink structure of the web. Specifically, 
Aq - j if page j links to page i; here dj is the number of distinct pages linked 
from page j. 

(2.) Find X satisfying X = e[l] + (1 - e)AX. Here e is a constant, e.g., and [1] 
represents a column vector of Is. The value X[i] is the rank of the zth page. 

The most commonly used approach to solving the above equation is to start with 
a value of X, where each component is £ (where n is the number of pages) and then 
perform the following iteration: X* = e[l] + (1 - e)AX k -\. The iteration stops when 
the X k converges, i.e., the difference from X k to X k+ i is less than a specified threshold. 

Design a system that can compute the ranks of ten billion web pages in a reasonable 
amount of time. 

Hint: This must be performed on an ensemble of machines. The right data structures will 
simplify the computation. 

Solution: Since the web graph can have billions of vertices and it is mostly a sparse 
graph, it is best to represent the graph as an adjacency list. Building the adjacency 
list representation of the graph may require a significant amount of computation. 


397 



depending upon how the information is collected. Usually, the graph is constructed 
by downloading the pages on the web and extracting the hyperlink information from 
the pages. Since the URL of a page can vary in length, it is often a good idea to 
represent the URL by a hash code. 

The most expensive part of the PageRank algorithm is the repeated matrix multi¬ 
plication. Usually, it is not possible to keep the entire graph information in a single 
machine's RAM. Two approaches to solving this problem are described below. 

• Disk-based sorting—we keep the column vector X in memory and load rows one 
at a time. Processing Row i simply requires adding A^Xj to Xy for each j such 
that A ir j is not zero. The advantage of this approach is that if the column vector 
fits in RAM, the entire computation can be performed on a single machine. This 
approach is slow because it uses a single machine and relies on the disk. 

• Partitioned graph—we use n servers and partition the vertices (web pages) into 
n sets. This partition can be computed by partitioning the set of hash codes in 
such a way that it is easy to determine which vertex maps to which machine. 
Given this partitioning, each machine loads its vertices and their outgoing 
edges into RAM. Each machine also loads the portion of the PageRank vector 
corresponding to the vertices it is responsible for. Then each machine does a 
local matrix multiplication. Some of the edges on each machine may correspond 
to vertices that are owned by other machines. Hence the result vector contains 
nonzero entries for vertices that are not owned by the local machine. At the end 
of the local multiplication it needs to send updates to other hosts so that these 
values can be correctly added up. The advantage of this approach is that it can 
process arbitrarily large graphs. 

PageRank runs in minutes on a single machine on the graph consisting of the multi¬ 
million pages that constitute Wikipedia. It takes roughly 70 iterations to converge on 
this graph. Anecdotally, PageRank takes roughly 200 iterations to converge on the 
graph consisting of the multi-billion pages that constitute the World-Wide Web. 

When operating on a graph the scale of the webgraph, PageRank must be run on 
many thousands of machines. In such situations, it is very likely that a machine will 
fail, e.g., because of a defective power supply. The widely used Map-Reduce frame¬ 
work handles efficient parallelization as well as fault-tolerance. Roughly speaking, 
fault-tolerance is achieved by replicating data across a distributed file system, and 
having the master reassign jobs on machines that are not responsive. 


21.9 Design TeraSort and PetaSort 

Modem datasets are huge. For example, it is estimated that a popular social network 
contains over two trillion distinct items. 

How would you sort a billion 1000 byte strings? How about a trillion 1000 byte 
strings? 

Hint: Can a trillion 1000 byte strings fit on one machine? 

Solution: A billion 1000 byte strings cannot fit in the RAM of a single machine, but can 
fit on the hard drive of a single machine. Therefore, one approach is to partition the 
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data into smaller blocks that fit in RAM, sort each block individually, write the sorted 
block to disk, and then combine the sorted blocks. The sorted blocks can be merged 
using for example Solution 11.1 on Page 177. The UNIX sort program uses these 
principles when sorting very large files, and is faster than direct implementations of 
the merge-based algorithm just described. 

If the data consists of a trillion 1000 byte strings, it cannot fit on a single machine— 
it must be distributed across a cluster of machines. The most natural interpretation 
of sorting in this scenario is to organize the data so that lookups can be performed 
via binary search. Sorting the individual datasets is not sufficient, since it does not 
achieve a global ordering—lookups entail a binary search on each machine. The 
straightforward solution is to have one machine merge the sorted datasets, but then 
that machine will become the bottleneck. 

A solution which does away with the bottleneck is to first reorder the data so that 
the ith machine stores strings in a range, e.g.. Machine 3 is responsible for strings that 
lie between daily and ending. The range-to-machine mapping R can be computed by 
sampling the individual files and sorting the sampled values. If the sampled subset is 
small enough, it can be sorted by a single machine. The techniques in Solution 12.8 on 
Page 200 can be used to determine the ranges assigned to each machine. Specifically, 
let A be the sorted array of sampled strings. Suppose there are M machines. Define 
Y{ - iA[n/M], where n is the length of A. Then Machine i is responsible for strings in 
the range [r„ r i+ 1 ). If the distribution of the data is known a priori, e.g., it is uniform, 
the sampling step can be skipped. 

The reordering can be performed in a completely distributed fashion by having 
each machine route the strings it begins with to the responsible machines. 

After reordering, each machine sorts the strings it stores. Consequently queries 
such as lookups can be performed by using R to determine which individual machine 
to forward the lookup to. 


21.10 Implement distributed throttling 

You have n machines ("crawlers") for downloading the entire web. The responsi¬ 
bility for a given URL is assigned to the crawler whose ID is Hash(URL) mod n. 

Downloading a web page takes away bandwidth from the web server hosting it. 

Implement crawling under the constraint that in any given minute your crawlers do 

not request more than b bytes from any website. 

Hint: Use a server to coordinate the crawl. 

Solution: This problem, as posed, is ambiguous. 

• Since we usually download one file in one request, if a file is greater than b bytes, 
there is no way we can meet the constraint of serving fewer than b bytes every 
minute, unless we can work with the lower layers of the network stack such 
as the transport layer or the network layer. Often the system designer could 
look at the distribution of file sizes and conclude that this problem happens so 
infrequently that we do not care. Alternatively, we may choose to download 
no more than the first b bytes of any file. 
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• Given that the host's bandwidth is a resource for which there could be con¬ 
tention, one important design choice to be made is how to resolve a contention. 
Do we let requests get served in first-come first-served order or is there a notion 
of priority? Often crawlers have a built-in notion of priority based on how 
important the document is to the users or how fresh the current copy is. 

One way of doing this could be to maintain a permission server with which each 
crawler checks to see if it is okay to hit a particular host. The server can keep an 
account of how many bytes have been downloaded from the server in the last minute 
and not permit any crawler to hit the server if we are already close to the quota. If we 
do not care about priority, then we can keep the interface synchronous where a server 
requests permission to download a file and it immediately gets approved or denied. 
If we care about priorities, then the server may enqueue the request and inform the 
crawler when the file is available for download. The queues at the permission server 
may be based on priorities. 

If the permission server becomes a bottleneck, we can use multiple permission 
servers such that the responsibility of a given host is decided by applying a hash 
function to the host name and assigning it to a particular server based on the hash 
code. 


21.11 Design a scalable priority system 

Maintaining a set of prioritized jobs in a distributed system can be tricky. Applications 
include a search engine crawling web pages in some prioritized order, as well as event- 
driven simulation in molecular dynamics. In both cases the number of jobs is in the 
billions and each has its own priority. 

Design a system for maintaining a set of prioritized jobs that implements the following 
API: (1.) insert a new job with a given priority; (2.) delete a job; (3.) find the highest 
priority job. Each job has a unique ID. Assume the set cannot fit into a single machine's 
memory. 

Hint: How would you partition jobs across machines? Is it always essential to operate on the 
highest priority job? 

Solution: If we have enough RAM on a single machine, the most simple solution 
would be to maintain a min-heap where entries are ordered by their priority. An 
additional hash table can be used to map jobs to their corresponding entry in the 
min-heap to make deletions fast. 

A more scalable solution entails partitioning the problem across multiple ma¬ 
chines. One approach is to apply a hash function to the job ids and partition the 
resulting hash codes into ranges, one per machine. Insert as well as delete require 
communication with just one server. To do extract-min, we send a lookup minimum 
message to all the machines, infer the min from their responses, and then delete it. 

At a given time many clients may be interested in the highest priority event, and 
it is challenging to distribute this problem well. If many clients are trying to do 
this operation at the same time, we may run into a situation where most clients will 
find that the min event they are trying to extract has already been deleted. If the 
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throughput of this service can be handled by a single machine, we can make one 
server solely responsible for responding to all the requests. This server can prefetch 
the top hundred or so events from each of the machines and keep them in a heap. 

In many applications, we do not need strong consistency guarantees. We want to 
spend most of our resources taking care of the highest priority jobs. In this setting, 
a client could pick one of the machines at random, and request the highest priority 
job. This would work well for the distributed crawler application. It is not suited to 
event-driven simulation because of dependencies. 

Another other issue to consider is resilience: if a node fails, all list of work on that 
node fails as well. It is better to have nodes to contain overlapped lists and the dis¬ 
patching node in this case will handle duplicates. The lost of a node shouldn't result 
in full re-hashing—the replacement node should handle only new jobs. Consistent 
hashing can be used to achieve this. 

A front-end caching server can become a bottleneck. This can be avoided by using 
replication, i.e., multiple servers which duplicate each other. There could be possible 
several ways to coordinate them: use non-overlapping lists, keep a blocked job list, 
return a random job from the jobs with highest priority. 


21.12 Create photomosaics 

A photomosaic is built from a collection of images called "tiles" and a target image. 
The photomosaic is another image which approximates the target image and is built 
by juxtaposing the tiles. Quality is defined by human perception. 

Design a program that produces high quality mosaics with minimal compute time. 

Hint: How would you define the distance between two images? 

Solution: A good way to begin is to partition the image into s x s-sized squares, 
compute the average color of each such image square, and then find the tile that is 
closest to it in the color space. Distance in the color space can be the L2-distance over 
the Red-Green-Blue (RGB) intensities for the color. As you look more carefully at the 
problem, you might conclude that it would be better to match each tile with an image 
square that has a similar structure. One way could be to perform a coarse pixelization 
(2 X 2 or 3 X 3) of each image square and finding the tile that is "closest" to the image 
square under a distance function defined over all pixel colors. In essence, the problem 
reduces to finding the closest point from a set of points in a /c-dimensional space. 

Given m tiles and an image partitioned into n squares, then a brute-force approach 
would have 0(mn) time complexity. You could improve on this by first indexing 
the tiles using an appropriate search tree. You can also run the matching in parallel 
by partitioning the original image into subimages and searching for matches on the 
subimages independently. 


21.13 Implement Mileage Run 

Airlines often give customers who fly frequently with them a "status". This status 
allows them early boarding, more baggage, upgrades to executive class, etc. Typically, 
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status is a function of miles flown in the past twelve months. People who travel 
frequently by air sometimes want to take a round trip flight simply to maintain their 
status. The destination is immaterial—the goal is to minimize the cost-per-mile (cpm), 
i.e., the ratio of dollars spent to miles flown. 

Design a system that will help its users find mileage runs. 

Hint: Partition the implied features into independent tasks. 

Solution: There are two distinct aspects to the design. The first is the user-facing 
portion of the system. The second is the server backend that gets flight-price-distance 
information and combines it with user input to generate the alerts. 

We begin with the user-facing portion. For simplicity, we illustrate it with a web- 
app, with the realization that the web-app could also be written as a desktop or mobile 
app. The web-app has the following components: a login page, a manage alerts page, 
a create an alert page, and a results page. For such a system we would like defer 
to a single-sign-on login service such as that provided by Google or Facebook. The 
management page would present login information, a list of alerts, and the ability to 
create an alert. 

One reasonable formulation of an alert is that it is an origin city, a target cpm, 
and optionally, a date or range of travel dates. The results page would show flights 
satisfying the constraints. Note that other formulations are also possible, such as how 
frequently to check for flights, a set of destinations, a set of origins, etc. 

The classical approach to implement the web-app front end is through dynamically 
generated HTML on the server, e.g., through Java Server Pages. It can be made more 
visually appealing and intuitive by making appropriate use of cascaded style sheets, 
which are used for fonts, colors, and placements. The UI can be made more efficient 
through the use of Javascript to autocomplete common fields, and make attractive 
date pickers. 

Modern practice is to eschew server-side HTML generation, and instead have 
a single-page application, in which Javascript reads and writes JavaScript Object 
Notation (JSON) objects to the server, and incrementally updates the single-page 
based. The AngularJS framework supports this approach. 

The web-app backend server has four components: gathering flight data, matching 
user-generated alerts to this data, persisting data and alerts, and generating the 
responses to browser initiated requests. 

Flight data can be gathered via "scraping" or by subscribing to a flight data service. 
Scraping refers to extraction of data from a website. It can be quite involved—some 
of the issues are parsing the results from the website, filling in form data, and running 
the Javascript that often populates the actual results on a page. Selenium is a Java 
library that can programmatically interface to the Firefox browser, and is appropriate 
for scraping sites that are rich in Javascript. Most flight data services are paid. ITA 
software provides a very widely used paid aggregated flight data feed service. The 
popular Kayak site provides an Extensible Markup Language (XML) feed of recently 
discovered fares, which can be a good free alternative. Flight data does not include the 
distance between airports, but there are websites which return the distance between 
airport codes which can be used to generate the cpm for a flight. 
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There are a number of common web application frameworks—essentially libraries 
that handle many common tasks—that can be used to generate the server. Java and 
Python are very commonly used for writing the backend for web applications. 

Persistence of data can be implemented through a database. Most web applica¬ 
tion frameworks provide support for automating the process of reading and writing 
objects from and to a database. Finally, web application frameworks can route incom¬ 
ing HTTP requests to appropriate code—this is through a configuration file matching 
URLs to methods. The framework provides convenience methods for accessing HTTP 
fields and writing results. Frameworks also provide HTTP templating mechanisms, 
wherein developers intersperse HTML with snippets of code that dynamically add 
content to the HTML. 

Web application frameworks typically implement cron functionality, wherein spec¬ 
ified functions are executed at a regular interval. This can be used to periodically 
scrape data and check if the condition of an alert is matched by the data. 

Finally, the web app can be deployed via a platform-as-a-service such as Amazon 
Web Services and Google App Engine. 


21.14 Implement Connexus 

How would you design Connexus, a system by which users can share pictures? 
Address issues around access control (public, private), picture upload, organizing 
pictures, summarizing sets of pictures, allowing feedback, and displaying geograph¬ 
ical and temporal information for pictures. 

Hint: Think about the UI, in particular UI widgets that make for an engaging product. 

Solution: There are three aspects to Connexus. The first is the server backend, used to 
store images, and meta-data, such as author, comments, hashtags, GPS coordinates, 
etc., as well as run cron jobs to identify trending streams. The technology for this 
is similar to Mileage Run (Solution 21.13 on the facing page), with the caveat that a 
database is not suitable for storing large binary objects such as images, which should 
be stored on a local or remote file system. (The database will hold references to the 
file.) 

The web UI is also similar to Mileage Run, with a login page, a management page, 
and pages for displaying images. Images can be grouped based on a concept of a 
stream, with comment boxes annotating streams. Facebook integration would make 
it easier to share links to streams, and post new images as status updates. Search 
capability and discussion boards also enhance the user experience. 

Some UI features that are especially appealing are displaying images by location 
on a zoomable map, slider UI controls to show subsets of images in a stream based on 
the selected time intervals, a file upload dialog with progress measures, support for 
multiple simultaneous uploads, and drag-and-drop upload. All these UI widgets are 
provided by, for example, the jQuery-UI Javascript library. (This library also makes 
the process of creating autocompletion on text entry fields trivial.) 

Connexus is an application that begs for a mobile client. A smartphone provides 
a camera, location information, and push notifications. These can be used to make it 
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easier to create and immediately upload geo-tagged images, find nearby images, and 
be immediately notified on updates and comments. The two most popular mobile 
platforms, iOS and Android, have APIs rich in UI widgets and media access. Both 
can use serialization formats such as JSON or protocol buffers to communicate with 
the server via remote procedure calls layered over HTTP. 


21.15 Design an online advertising system 

Jingle, a search engine startup, has been very successful at providing a high-quality 
Internet search service. A large number of customers have approached Jingle and 
asked it to display paid advertisements for their products and services alongside 
search results. 

Design an advertising system for Jingle. 

Hint: Consider the stakeholders separately. 

Solution: Reasonable goals for such a system include 

• providing users with the most relevant ads, 

• providing advertisers the best possible return on their investment, and 

• minimizing the cost and maximizing the revenue to Jingle. 

Two key components for such a system are: 

• The front-facing component, which advertisers use to create advertisements, 
organize campaigns, limit when and where ads are shown, set budgets, and 
create performance reports. 

• The ad-serving system, which selects which ads to show on the searches. 

The front-facing system can be a fairly conventional web application, i.e., a set of 
web pages, middleware that responds to user requests, and a database. Key features 
include: 

• User authentication—a way for users to create accounts and authenticate them¬ 
selves. Alternatively, use an existing single sign-on login service, e.g., Facebook 
or Google. 

• User input—a set of form elements to let advertisers specify ads, advertising 
budget, and search keywords to bid on. 

• Performance reports—a way to generate reports on how the advertiser's money 
is being spent. 

• Customer service—even the best of automated systems require occasional hu¬ 
man interaction, e.g., ways to override limits on keywords. This requires an 
interface for advertisers to contact customer service representatives, and an 
interface for those representatives to interact with the system. 

The whole front-end system can be built using, for example, HyperText Markup 
Language (HTML) and JavaScript. A commonly used approach is to use a LAMP 
stack on the server-side: Linux as the OS, Apache as the HTTP server, MySQL as the 
database software, and PHP for the application logic. 

The ad-serving system is less conventional. The ad-serving system would build a 
specialized data structure, such as a decision tree, from the ads database. It chooses 
ads from the database of ads based on their "relevance" to the search. In addition 
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to keywords, the ad-serving systems can use knowledge of the user's search history, 
how much the advertiser is willing to pay, the time of day, user locale, and type of 
browser. Many strategies can be envisioned here for estimating relevance, such as, 
using information retrieval or machine learning techniques that learn from past user 
interactions. 

The ads could be added to the search results by embedding JavaScript in the results 
page. This JavaScript pulls in the ads from the ad-serving system directly. This helps 
isolate the latency of serving search results from the latency of serving ad results. 

There are many more issues in such a system: making sure there are no inappro¬ 
priate images using an image recognition API; using a link verification to check if 
keywords really corresponds to a real site; serving up images from a content-delivery 
network; and having a fallback advertisement to show if an advertisement cannot be 
found. 


21.16 Design a recommendation system 

Jingle wants to generate more page views on its news site. A product manager has 
the idea to add to each article a sidebar of clickable snippets from articles that are 
likely to be of interest to someone reading the current article. 

Design a system that automatically generates a sidebar of related articles. 

Hint: This problem can be solved with various degrees of algorithmic sophistication: none at 
all, simple frequency analysis, or machine learning. 

Solution: The key technical challenge in this problem is to come up with the list of 
articles—the code for adding these to a sidebar is trivial. 

One suggestion might be to add articles that have proved to be popular recently. 
Another is to have links to recent news articles. A human reader at Jingle could tag 
articles which he believes to be significant. He could also add tags such as finance, 
sports, and politics, to the articles. These tags could also come from the HTML 
meta-tags or the page title. 

We could also provide randomly selected articles to a random subset of readers 
and see how popular these articles prove to be. The popular articles could then be 
shown more frequently. 

On a more sophisticated level. Jingle could use automatic textual analysis, where 
a similarity is defined between pairs of articles—this similarity is a real number and 
measures how many words are common to the two. Several issues come up, such as 
the fact that frequently occurring words such as "for" and "the" should be ignored 
and that having rare words such as "arbitrage" and "diesel" in common is more 
significant than having say, "sale" and "international". 

Textual analysis has problems, such as the fact that two words may have the same 
spelling but completely different meanings (anti-virus means different things in the 
context of articles on acquired immune deficiency syndrome (AIDS) and computer 
security). One way to augment textual analysis is to use collaborative filtering— 
using information gleaned from many users. For example, by examining cookies and 
timestamps in the web server's log files, we can tell what articles individual users 
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have read. If we see many users have read both A and B in a single session, we might 
want to recommend B to anyone reading A. For collaborative filtering to work, we 
need to have many users. 


21.17 Design an optimized way of distributing large files 

Jingle is developing a search feature for breaking news. New articles are collected 
from a variety of online news sources such as newspapers, bulletin boards, and blogs, 
by a single lab machine at Jingle. Every minute, roughly one thousand articles are 
posted and each article is 100 kilobytes. 

Jingle would like to serve these articles from a data center consisting of a 1000 
servers. For performance reasons, each server should have its own copy of articles 
that were recently added. The data center is far away from the lab machine. 

Design an efficient way of copying one thousand files each 100 kilobytes in size from 
a single lab server to each of 1000 servers in a distant data center. 

Hint: Exploit the data center. 

Solution: Assume that the bandwidth from the lab machine is a limiting factor. It 
is reasonable to first do trivial optimizations, such as combining the articles into a 
single file and compressing this file. 

Opening 1000 connections from the lab server to the 1000 machines in the data 
center and transferring the latest news articles is not feasible since the total data 
transferred will be approximately 100 gigabytes (without compression). 

Since the bandwidth between machines in a data center is very high, we can copy 
the file from the lab machine to a single machine in the data center and have the 
machines in the data center complete the copy. Instead of having just one machine 
serve the file to the remaining 999 machines, we can have each machine that has 
received the file initiate copies to the machines that have not yet received the file. In 
theory, this leads to an exponential reduction in the time taken to do the copy. 

Several additional issues have to be dealt with. Should a machine initiate further 
copies before it has received the entire file? (This is tricky because of link or server 
failures.) How should the knowledge of machines which do not yet have copies of 
the file be shared? (There can be a central repository or servers can simply check 
others by random selection.) If the bandwidth between machines in a data center is 
not a constant, how should the selections be made? (Servers close to each other, e.g., 
in the same rack, should prefer communicating with each other.) 

Finally, it should be mentioned that there are open source solutions to this problem, 
e.g.. Unison and BitTorrent, which would be a good place to start. 


21.18 Design the World Wide Web 

Design the World Wide Web. Specifically, describe what happens when you enter a 
URL in a browser address bar, and press return. 

Hint: Follow the flow of information. 
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Solution: At the network level, the browser extracts the domain name component 
of the URL, and determines the IP address of the server, e.g., through a call to a 
Domain Name Server (DNS), or a cache lookup. It then communicates using the 
HTTP protocol with the server. HTTP itself is built on top of TCP/IP, which is 
responsible for routing, reassembling, and resending packets, as well as controlling 
the transmission rate. 

The server determines what the client is asking for by looking at the portion of 
the URL that comes after the domain name, and possibly also the body of the HTTP 
request. The request may be for something as simple a file, which is returned by 
the Webserver; HTTP spells out a format by which the type of the returned file is 
specified. For example, the URL http://go.com/imgs/abc.png may encode a request for 
the file whose hierarchical name is imgs/abc.png relative to a base directory specified 
at configuration to the web server. 

The URL may also encode a request to a service provided by the web server. For 
example, http://go.com/lookup/flight?num=UA37,city=AUS is a request to the lookup/flight 
service, with an argument consisting of two attribute-value pair. The service could be 
implemented in many ways, e.g., Java code within the server, or a Common Gateway 
Interface (CGI) script written in Perl. The service generates a HTTP response, typically 
HTML, which is then returned to the browser. This response could encode data which 
is used by scripts running in the browser. Common data formats include JSON and 
XML. 

The browser is responsible for taking the returned HTML and displaying it on the 
client. The rendering is done in two parts. First, a parse tree (the DOM) is generated 
from the HTML, and then a rendering library "paints" the screen. The returned 
HTML may include scripts written in JavaScript. These are executed by the browser, 
and they can perform actions like making requests and updating the DOM based on 
the responses—this is how a live stock ticker is implemented. Styling attributes (CSS) 
are commonly used to customize the look of a page. 

Many more issues exist on both the client and server side: security, cookies, HTML 
form elements, HTML styling, and handlers for multi-media content, to name a few. 


21.19 Estimate the hardware cost of a photo sharing app 

Estimate the hardware cost of the server hardware needed to build a photo sharing 
app used by every person on the earth. 

Hint: Use variables to denote quantities and costs, and relate them with equations. Then fill in 
reasonable values. 

Solution: The cost is a function of server CPU and RAM resources, storage, and 
bandwidth, as well as the number and size of the images that are uploaded each 
day. We will create an estimate based on unit costs for each of these. We assume a 
distributed architecture in which images are spread ("sharded") across servers. 

Assume each user uploads i images each day with an average size of s bytes, and 
that each image is viewed v times. After d days, the storage requirement is isdN, 
where N is the number of users. Assuming v » 1, i.e., most images are seen many 
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times, the server cost is dominated by the time to serve the images. The servers 
are required to serve up Niv images and Nivs bytes each day. Assuming a server 
can handle h HTTP requests per second and has an outgoing bandwidth of b bytes 
per second, the number of required servers is ma x(Niv/Th,Nivs/Tb), where T is the 
number of seconds in a day. 

Reasonable values for N, i, s, and v are 10 10 , 10,10 5 , and 100. Reasonable values 
for h and b are 10 4 and 10 8 . There are approximately 10 5 seconds in a day. Therefore 
the number of servers required is max((10 10 X 10 X 100)/(10 5 X 10 4 ), (10 10 X 10 X 100 X 
10 5 )/(10 5 X 10 8 )) = 10 5 . Each server would cost $1000, so the total cost would be of 
the order of 100 million dollars. 

Storage costs are approximately $0.1 per gigabyte, and we add Nis = 10 10 X10 X10 5 
bytes each day, so each day we need to add a million dollars worth of storage. 

The above calculation leaves out many costs, such as electricity, cooling, and net¬ 
work. It also neglects computations such as computing trending data and spam 
analysis. Furthermore, there is no measure of the cost of redundancy, such as repli¬ 
cated storage, or the ability to handle nonuniform loads. Nevertheless, it is a decent 
starting point. What is remarkable is the fact that the entire world can be connected 
through images at a very low cost—pennies per person. 
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Chapter 


Language Questions 


The limits of my language means the limits of my world. 


— L. Wittgenstein 


We now present some commonly asked problems on the Java language. This set is 
not meant to be comprehensive—it's a test of preparedness. If you have a hard time 
with them, we recommend that you spend more time reviewing language basics. 

22.1 The JVM 

Explain what a JVM is, and what its benefits and drawbacks are. 

Solution: A Java program is compiled into bytecode, not machine language (as is 
the case for example for a C or C++ program). Bytecode is a platform independent 
instruction set that is executed by an interpreter, which itself is a program, usually 
written in C++ or C. The JVM is the interpreter for the bytecode generated by the 
Java compiler. Here are the key advantages and disadvantages of this approach: 

• The same bytecode can run unchanged on multiple platforms. (Sun's marketing 
team referred to this as "write once, run everywhere".) This approach also 
greatly reduces the burden of porting a language to a new platform—all that is 
needed is to implement the interpreter and interface key libraries, e.g., for I/O. 

• The primary drawback of using a JVM is the performance hit brought about by 
the added layer of indirection. Recently, this performance penalty has reduced 
greatly through the use of just-in-time-compilation, wherein the JVM identifies 
at run-time code sections that are most commonly executed, e.g., loops, and 
generates native machine code for those sections. (Occasionally, because the 
JVM can use runtime profiling it can better predict branches, which leads to 
performance that is actually superior to a statically compiled language.) 

Variant: Describe a way in which a program written in Java can leak memory. (Item 6, 
Effective Java) 


22.2 THROW VS. THROWS 

Explain the difference between throw and throws. 

Solution: Both are keywords related to exception handling. 
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• The throw keyword is used to throw an exception from a method or static 
block, e.g., if (A.length == 0) throw IllegalArgumentException("Array 
must not be empty"); 

• The throws keyword is used in method declaration to indicate exceptions that 
can be possibly be thrown by the method, e.g., public void binSearch(int [] 
A, int k) throws IllegalArgumentException. A method that throws 
checked exceptions (e.g., FileNotFoundException) must declare all of them. 
Unchecked exceptions, e.g., IllegalArgumentException, that are thrown by 
the method may be declared optionally. 

Variant: What is the difference between checked and unchecked exceptions? List five 
commonly used standard exceptions. (Item 58, Effective Java) 


22.3 FINAL, FINALLY, AND FINALIZER 

Explain the difference between final, finally, and finalizer. 

Solution: All three are keywords. Despite their textual similarity, they are very 
different: 

• final is commonly used to ensure a variable is assigned to just once. (This 
assignment can take place where it is declared, or in the constructor.) It is also 
used when declaring a method (static or nonstatic) to indicate that the method 
cannot be overridden. Finally, if a class is declared to be final, it means that it 
cannot be subclassed. Violating any of these uses of final is a compile-time 
error. 

• finally is used with a try-catch block to specify code that must be executed, 
regardless of whether an exception was thrown and/or caught. It is commonly 
used to prevent resource leaks when an exception is thrown, e.g., files not being 
closed, locks not being released, etc. 

• finalizer is nonstatic method that every class inherits from Ob j ect. It is called 
when the object is garbage collected. Ostensibly, it can be used to avoid leaks 
by explicitly closing files, nulling out object references, etc. In practice, since the 
exact time when an object is garbage collected is not under programmer control, 
these resources should be explicitly deallocated—an overridden finalizerO 
is a code smell. (Other drawbacks of finalizerO include low performance as 
well as unexpected behaviors on uncaught exceptions.) 


22.4 equals O vs. == 

Compare and contrast equals() and ==. 

Solution: First of all, this question only makes sense for reference types—primitive 
types can only be compared using ==, which does pretty much what you'd expect. 

A variable of a reference type holds an address. If x and y are variables of a 
reference type, then x--y tests if they hold the same address. 


410 



Very often, it is the case that objects that are located at distinct places in memory 
have the same content. In this case, we want to use logical equality. Every object inher¬ 
its a equals (Object o) method from the Object class. The default implementation 
is the same as ==. However, it is typically overridden to test logical equality. 

The usual recipe is to first test if == holds (to boost performance), then use 
instance of to see if the argument has the right type, then cast the argument to the 
correct type, and finally, for each "significant" field check if the field of the argument 
matches this' corresponding field. 

It's fine to rely on == in some cases, most notably when the class ensures at most 
one object exists with a given value. 


22.5 EQUALS() AND HASHCODEC) 

Why should you override hashCode () when you override equals()? 

Solution: As discussed in Problem 22.4 on the preceding page, it is appropriate to 
override equals() when objects distinct in memory may be logically equal. 

Every object inherits int hashCode() from Object. The default implementation 
uses the address of the object to compute the hashcode. Suppose a class A overrides 
equals() but not hashCode(). Let x be added to a HashSet<A> container. Suppose 
y is distinct but logically equivalent to x, i.e., x.equals(y) returns true. A call to 
contains (y) in the container will use y. hashCode () to select the hash bucket to 
search for y in, which in all likelihood be different from the bucket x was placed in, so 
the call returns false. (Even if the bucket was the same, the container's implementation 
caches the hashcode as a performance optimization, and will return still false.) This 
problem arises whenever objects are stored in other hash-based collections such as 
HashMap. 

The solution is to override hashCode () to look at each field that is read in equals (). 
Note that this principle has to be applied recursively, e.g., if fields are compared using 
their own equals(). 


22.6 List, ArrayList, and LinkedList 

Explain the difference between List, ArrayList, and LinkedList. 

Solution: All three are types that are used to represent sequences. 

• List is an interface—a definition that specifies method signatures, but no im¬ 
plementation of these methods. There are many benefits of using interfaces over 
classes, such as polymorphism, decoupling implementation from specification, 
and to specify mixins. 

• ArrayList is a class that implements the List interface. It is based on ar¬ 
rays. It has a couple of methods outside of those in the List interface, such 
as trimToSizeO, but essentially it just implements List. Because it is imple¬ 
mented using an array, testing and setting elements is fast (0(1)), but adding 
elements to the beginning is slow (0(n) for an array of n elements). (Adding 
elements to the end is fast because arrays overallocate buffer space.) 
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• LinkedList is an implementation of the List interface. It is based on the linked 
list data structure. It has many methods in addition to the List interface, 
e.g., it implements stack and queue functionality. For a linked list LinkedList 
insertion at the head or tail takes 0(1) time, but getting or setting the A:th element 
takes 0(k) time. 

Variant: Contrast the following code snippets. (Item 25, Effective Java) 

Object [] numArray = new Integer[2]; 

numArray[®] = 42; 

numArray[1] = "Hello World!"; 


List<Object> numArrayList = new ArrayList<Integer>(l); 
numArrayList.add(42); 


22.7 String vs. StringBuilder 

Compare and contrast String and StringBuilder. 

Solution: String is Java's built-in class for representing and operating on strings. It 
has a variety of constructors, and methods, e.g., for testing substring inclusion, ex¬ 
tracting substrings, etc. A key feature of Java string objects is that they are immutable. 
This has many benefits, e.g., 

• threads can safely share strings without fear of data corruption through races, 

• strings can be cached, i.e., an operation can return a reference to an existing 
string, and 

• strings can be safely added to Sets and Maps without fear of changing content 
corrupting the hashtable. 

The drawback of string immutability is that every time a string has to be modified, 
a new object has to be created, even for something as simple as changing the first 
character. If the string is long, this is a time-consuming operation. For example, 
consider the following program. 

public static String concat(List<String> strings) { 

String result = 
for (String s : strings) { 
result += s; 

} 

return result; 

} 


Every update to the result entails creating a new string and copying over the existing 
characters in the result to the start of the new string. If the string has length n this 
takes 0(n) time. Since there are n iterations, the time complexity grows as 0(n 2 ). 

The StringBuilder class is mutable. Additionally, the constructor overallocates 
space, which means that adding characters to the end is efficient (until the buffer fills 
up). The following program is functionally equivalent to the one above, but has 0(n) 
time complexity. 


412 



public static String concat(List<String> strings) { 
StringBuilder result = new StringBuilder(); 
for (String s : strings) { 
result.append(s) ; 

} 

return result.toString() ; 


22.8 Autoboxing 

What does autoboxing refer to? 

Solution: Types in Java are classified into two categories—primitive types and ref¬ 
erence types. Leaving aside long/short variants, primitive types are integers, chars, 
floats, and booleans. All remaining types are reference types—this includes arrays, 
classes, and interfaces. Variable of these type hold addresses. 

Nonstatic methods can only be called on reference types—there simply isn't 
enough space in a primitive type to store the dispatch table needed to call nonstatic 
methods. This means that primitive types cannot be directly added, for example, to 
hash tables or linked lists, since we need to be able to call hashCode () or equals () to 
determine where to put a value, or if a value is already present. 

Java uses box-types, e.g.. Integer to wrap primitive types, thereby allowing them 
to be used where objects are expected. 

Prior to Java 1.5, conversion between primitive types and corresponding box types 
was performed by code like X = new Integer(x) ; and x = Integer. intValue(X) ;, 
where x is of type int and X is of type Integer. Java 1.5 introduced the concept of 
autoboxing and auto-unboxing—the compiler takes an assignment like Integer X = 
x;, where x is assumed to be of type int and internally translates it to Integer 
X = Integer .valueOf(x); The assignment int x = X; internally becomes x = 
Integer.intValue(X);. 

Though autoboxing leads to cleaner code, and possibly reducing the number 
of allocated objects (since Integer .valueOfO can return cached values), it has its 
disadvantages too. Here are some examples. Autoboxing hides the performance 
hit brought about by object allocation. It's easy to use == to compare values where 
equals() is more appropriate. Since Integers can be null, int x = X; may throw 
NullPointerException. Conversions, e.g., from Integer to Long don't work as you 
might expect. 


22.9 Static initialization 
What is a static initializer block? 

Solution: Static variables are often initialized by one-line assignments, e.g., static 
int capacity = 16;, or static Foo bar = readFooFromConfigFile(). Occasion¬ 
ally, the initialization logic is more complex, and depends on the values of other static 
fields. In this case, a static initialization block is appropriate, e.g.. 
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private static final Map<String, Integer> monthToOrdinal = new HashMap<>(); 

// Static initializer block. Executed once, when class is first acccessed. 

static { 

int ordinal = ®; 

String[] months = {"January", "February", "March", "April", 

"May", "June", "July", "August", 

"September", "October", "November", "December"}; 

for (String month : months) { 

monthToOrdinal.put(month, ordinal++); 

} 

} 


In this case, we can replace the initializer block with a call to a static function. How¬ 
ever, when multiple static fields are being initialized, using a static function can lead 
to performance inefficiency and/or code duplication compared to a static block, e.g., 
imagine the following program, (Item 5, Effective Java), without a static initializer. 

private static final Date BOOM_START; 
private static final Date BOOM_END; 

static { 

Calendar gmtCal = Calendar.getlnstance(TimeZone.getTimeZone("GMT")); 
gmtCal.set(1946, Calendar.JANUARY, 1, ®, ®, ®) ; 

B00M_START = gmtCal.getTime(); 

gmtCal.set(1965, Calendar.JANUARY, 1, ®, ®, ®); 

BOOM_END = gmtCal.getTime(); 

} 


One drawback of static initialization blocks is that they are called when the class 
is first accessed, either to create an instance, or to access a static method or field. This 
order dependency can lead to unanticipated behaviors. 

Variant: Explain the difference between a static and a nonstatic inner class. (Item 22, 
Effective Java) 
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Chapter 


Object-Oriented Design 

One thing expert designers know not to do is solve every problem from 
first principles. 

— "Design Patterns: Elements of Reusable Object-Oriented Software ," 
E. Gamma, R. Helm, R. E. Johnson, and J. M. Vlissides, 1994 


A class is an encapsulation of data and methods that operate on that data. Classes 
match the way we think about computation. They provide encapsulation, which 
reduces the conceptual burden of writing code, and enable code reuse, through 
the use of inheritance and polymorphism. However, naive use of object-oriented 
constructs can result in code that is hard to maintain. 

A design pattern is a general repeatable solution to a commonly occurring problem. 
It is not a complete design that can be coded up directly—rather, it is a description of 
how to solve a problem that arises in many different situations. In the context of object- 
oriented programming, design patterns address both reuse and maintainability. In 
essence, design patterns make some parts of a system vary independently from the 
other parts. 

Adnan's Design Pattern course material, available freely online, contains lecture 
notes, homeworks, and labs that may serve as a good resource on the material in this 
chapter. 

23.1 Template Method vs. Strategy 

Explain the difference between the template method pattern and the strategy pattern 
with a concrete example. 

Solution: Both the template method and strategy patterns are similar in that both are 
behavioral patterns, both are used to make algorithms reusable, and both are general 
and very widely used. However, they differ in the following key way: 

• In the template method, a skeleton algorithm is provided in a superclass. Sub¬ 
classes can override methods to specialize the algorithm. 

• The strategy pattern is typically applied when a family of algorithms imple¬ 
ments a common interface. These algorithms can then be selected by clients. 

As a concrete example, consider a sorting algorithm like quicksort. Two of the key 
steps in quicksort are pivot selection and partitioning. Quicksort is a good example of 
a template method—subclasses can implement their own pivot selection algorithm, 
e.g., using randomized median finding or selecting an element at random, and their 
own partitioning method, e.g., using the DNF partitioning algorithms in Solution 6.1 
on Page 63. 
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Since there may be multiple ways in which to sort elements, e.g., student objects 
may be compared by GPA, major, name, and combinations thereof, it's natural to 
make the comparision operation used by the sorting algorithm an argument to quick¬ 
sort. One way to do this is to pass quicksort an object that implements a compare 
method. These objects constitute an example of the strategy pattern, as do the objects 
implementing pivot selection and partitioning. 

There are some other smaller differences between the two patterns. For example, 
in the template method pattern, the superclass algorithm may have "hooks"—calls 
to placeholder methods that can be overridden by subclasses to provide additional 
functionality. Sometimes a hook is not implemented, thereby forcing the subclasses to 
implement that functionality; some times it offers a "no-operation" or some baseline 
functionality. There is no analog to a hook in a strategy pattern. 

Note that there's no relationship between the template method pattern and tem¬ 
plate meta-programming (a form of generic programming favored in C++). 


23.2 Observer pattern 

Explain the observer pattern with an example. 

Solution: The observer pattern defines a one-to-many dependency between objects 
so that when one object changes state all its dependents are notified and updated 
automatically. 

The observed object must implement the following methods. 

• Register an observer. 

• Remove an observer. 

• Notify all currently registered observers. 

The observer object must implement the following method. 

• Update the observer. (Update is sometimes referred to as notify.) 

As a concrete example, consider a service that logs user requests, and keeps track 
of the 10 most visited pages. There may be multiple client applications that use this 
information, e.g., a leaderboard display, ad placement algorithms, recommendation 
system, etc. Instead of having the clients poll the service, the service, which is the 
observed object, provides clients with register and remove capabilities. As soon as 
its state changes, the service enumerates through registered observers, calling each 
observer's update method. 

Though most readily understood in the context of a single program, where the ob¬ 
served and observer are objects, the observer pattern is also applicable to distributed 
computing. 


23.3 Push vs. pull observer pattern 

In the observer pattern, subjects push information to their observers. There is another 
way to update data—the observers may "pull" the information they need from the 
subject. Compare and contrast these two approaches. 

Solution: Both push and pull observer designs are valid and have tradeoffs de¬ 
pending on the needs of the project. With the push design, the subject notifies the 
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observer that its data is ready and includes the relevant information that the observer 
is subscribing to, whereas with the pull design, it is the observer's job to retrieve that 
information from the subject. 

The pull design places a heavier load on the observers, but it also allows the 
observer to query the subject only as often as is needed. One important consideration 
is that by the time the observer retrieves the information from the subject, the data 
could have changed. This could be a positive or negative result depending on the 
application. The pull design places less responsibility on the subject for tracking 
exactly which information the observer needs, as long as the subject knows when to 
notify the observer. This design also requires that the subject make its data publicly 
accessible by the observers. This design would likely work better when the observers 
are running with varied frequency and it suits them best to get the data they need on 
demand. 

The push design leaves all of the information transfer in the subject's control. The 
subject calls update for each observer and passes the relevant information along with 
this call. This design seems more object-oriented, because the subject is pushing its 
own data out, rather than making its data accessible for the observers to pull. It is 
also somewhat simpler and safer in that the subject always knows when the data is 
being pushed out to observers, so you don't have to worry about an observer pulling 
data in the middle of an update to the data, which would require synchronization. 


23.4 Singletons and Flyweights 

Explain the differences between the singleton pattern and the flyweight pattern. Use 
concrete examples. 

Solution: The singleton pattern ensures a class has only one instance, and provides a 
global point of access to it. The flyweight pattern minimizes memory use by sharing 
as much data as possible with other similar objects. It is a way to use objects in large 
numbers when a simple repeated representation would use an unacceptable amount 
of memory. 

A common example of a singleton is a logger. There may be many clients who 
want to listen to the logged data (console, file, messaging service, etc.), so all code 
should log to a single place. 

A common example of a flyweight is string interning—a method of storing only 
one copy of each distinct string value. Interning strings makes some string processing 
tasks more time- or space-efficient at the cost of requiring more time when the string 
is created or interned. The distinct values are usually stored in a hash table. Since 
multiple clients may refer to the same flyweight object, for safety flyweights should 
be immutable. 

There is a superficial similarity between singleton and flyweight: both keep a 
single copy of an object. There are several key differences between the two: 

• Flyweights are used to save memory. Singletons are used to ensure all clients 
see the same object. 

• A singleton is used where there is a single shared object, e.g., a database con¬ 
nection, server configurations, a logger, etc. A flyweight is used where there 
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is a family of shared objects, e.g., objects describing character fonts, or nodes 
shared across multiple binary search trees. 

• Flyweight objects are invariable immutable. Singleton objects are usually not 
immutable, e.g., requests can be added to the database connection object. 

• The singleton pattern is a creational pattern, whereas the flyweight is a structural 
pattern. 

In summary, a singleton is like a global variable, whereas a flyweight is like a pointer 
to a canonical representation. 

Sometimes, but not always, a singleton object is used to create flyweights—clients 
ask the singleton for an object with specified fields, and the singleton checks its 
internal pool of flyweights to see if one exists. If such an object already exists, it 
returns that, otherwise it creates a new flyweight, add it to its pool, and then returns 
it. (In essence the singleton serves as a gateway to a static factory.) 


23.5 Adapters 

What is the difference between a class adapter and an object adapter? 

Solution: The adapter pattern allows the interface of an existing class to be used from 
another interface. It is often used to make existing classes work with others without 
modifying their source code. 

There are two ways to build an adapter: via subclassing (the class adapter pattern) 
and composition (the object adapter pattern). In the class adapter pattern, the adapter 
inherits both the interface that is expected and the interface that is pre-existing. In 
the object adapter pattern, the adapter contains an instance of the class it wraps and 
the adapter makes calls to the instance of the wrapped object. 

Here are some remarks on the class adapter pattern. 

• The class adapter pattern allows re-use of implementation code in both the 
target and adaptee. This is an advantage in that the adapter doesn't have to 
contain boilerplate pass-throughs or cut-and-paste reimplementation of code 
in either the target or the adaptee. 

• The class adapter pattern has the disadvantages of inheritance (changes in 
base class may cause unforeseen misbehaviors in derived classes, etc.). The 
disadvantages of inheritance are made worse by the use of two base classes, 
which also precludes its use in languages like Java (prior to Java 1.8) that do 
not support multiple inheritance. 

• The class adapter can be used in place of either the target or the adaptee. This 
can be advantage if there is a need for a two-way adapter. The ability to 
substitute the adapter for adaptee can be a disadvantage otherwise as it dilutes 
the purpose of the adapter and may lead to incorrect behavior if the adapter is 
used in an unexpected manner. 

• The class adapter allows details of the behavior of the adaptee to be changed 
by overriding the adaptee's methods. Class adapters, as members of the class 
hierarchy, are tied to specific adaptee and target concrete classes. 

As a concrete example of an object adapter, suppose we have legacy code that 
returns objects of type stack. Newer code expects inputs of type deque, which is 
more general than stack (but does not subclass stack). We could create a new type. 
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stack-adapter, which implements the deque methods, and can be used anywhere 
deque is required. The stack-adapter class has a field of type stack—this is referred 
to as object composition. It implements the deque methods with code that uses 
methods on the composed stack object. Deque methods that are not supported by 
the underlying stack throw unsupported operation exceptions. In this scenario, the 
stack-adapter is an example of an object adapter. 

Here are some comments on the object adapter pattern. 

• The object adapter pattern is "purer" in its approach to the purpose of making 
the adaptee behave like the target. By implementing the interface of the target 
only, the object adapter is only useful as a target. 

• Use of an interface for the target allows the adaptee to be used in place of any 
prospective target that is referenced by clients using that interface. 

• Use of composition for the adaptee similarly allows flexibility in the choice of 
the concrete classes. If adaptee is a concrete class, any subclass of adaptee will 
work equally well within the object adapter pattern. If adaptee is an interface, 
any concrete class implementing that interface will work. 

• A disadvantage is that if target is not based on an interface, target and all its 
clients may need to change to allow the object adapter to be substituted. 

Variant: The UML diagrams for decorator, adapter, and proxy look identical, so why 
is each considered a separate pattern? 


23.6 Creational Patterns 

Explain what each of these creational patterns is: builder, static factory, factory 
method, and abstract factory. 

Solution: The idea behind the builder pattern is to build a complex object in phases. 
It avoids mutability and inconsistent state by using an mutable inner class that has a 
build method that returns the desired object. Its key benefits are that it breaks down 
the construction process, and can give names to steps. Compared to a constructor, it 
deals far better with optional parameters and when the parameter list is very long. 
For example, the MapMaker class for caching in the Guava libarary makes it very clear 
how the cache is organized. 

A static factory is a function for construction of objects. Its key benefits are as 
follow: the function's name can make what it's doing much clearer compared to a call 
to a constructor. The function is not obliged to create a new object—in particular, it 
can return a flyweight. It can also return a subtype that's more optimized, e.g., it can 
choose to construct an object that uses an integer in place of a Boolean array if the array 
size is not more than the integer word size. The method Integer. valueOf (“123”) 
is a good example of a static factory—it caches values in the range [-128,127] that are 
already exist, thereby reducing memory footprint and construction time. 

A factory method defines interface for creating an object, but lets subclasses decide 
which class to instantiate. The classic example is a maze game with two modes—one 
with regular rooms, and one with magic rooms. The program below uses a template 
method, as described in Problem 23.1 on Page 415, to combine the logic common to 
the two versions of the game. 
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public abstract class MazeGame { 

// Template method pattern - uses subclasses implemention of makeRoom() 
public MazeGame() { 

Room rooml = makeRoom(); 

Room room2 = makeRoom(); 
rooml.connect(room2); 
this .addRoom(rooml); 
this .addRoom(room2); 

} 

// Implementing classes provide this method. 
abstract protected Room makeRoomQ ; 


This snippet implements the regular rooms. 


public class OrdinaryMazeGame extends MazeGame { 

// The default constructor will call superclass constructor. 
©Override 

protected Room makeRoom() { 
return new OrdinaryRoomQ ; 

} 

} 


This snippet implements the magic rooms. 


public class MagicMazeGame extends MazeGame { 

// The default constructor will call superclass constructor. 
©Override 

protected Room makeRoomQ { 
return new MagicRoomO; 

} 

} 


Here's how you use the factory to create regular and magic games. 

MazeGame ordinaryGame = new OrdinaryMazeGame(); 

MazeGame magicGame = new MagicMazeGame(); 

A drawback of the factory method pattern is that it makes subclassing challenging. 

An abstract factory provides an interface for creating families of related objects 
without specifying their concrete classes. For example, a class DocumentCreator 
could provide interfaces to create a number of products, such as createLetterO 
and createResume(). Concrete implementations of this class could choose to imple¬ 
ment these products in different ways, e.g., with modern or classic fonts, right-flush 
or right-ragged layout, etc. Client code gets a DocumentCreator object and calls its 
factory methods. Use of this pattern makes it possible to interchange concrete imple¬ 
mentations without changing the code that uses them, even at runtime. The price for 
this flexibility is more planning and upfront coding, as well as and code that may be 
harder to understand, because of the added indirections. 
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23.7 Libraries and design patterns 


Why is there no library of design patterns so a developers do not have to write code 
every time they want to use them? 

Solution: There are several reasons that design patterns cannot be delivered as a 
set of libraries. The key idea is that patterns cannot be cleanly abstracted from the 
objects and the processes they are applicable to. More specifically, libraries provide 
the implementations of algorithms. In contrast, design patterns provide a higher 
level understanding of how to structure classes and objects to solve specific types 
of problems. Another difference is that it's often necessary to use combinations of 
different patterns to solve a problem, e.g., Model-View-Controller (MVC), which is 
commonly used in UI design, incorporates the Observer, Strategy, and Composite 
patterns. It's not reasonable to come up with libraries for every possible case. 

Of course, many libraries take advantage of design patterns in their implemen¬ 
tations: sorting and searching algorithms use the template method pattern, custom 
comparision functions illustrate the strategy pattern, string interning is an example 
of the flyweight pattern, typed-I/O shows off the decorator pattern, etc. 

Variant: Give examples of commonly used library code that use the following pat¬ 
terns: template, strategy, observer, singleton, flyweight, static factory, decorator, 
abstract factory. 

Variant: Compare and contrast the iterator and composite patterns. 
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Chapter 


Common Tools 

UNIX is a general-purpose, multi-user, interactive oper¬ 
ating system for the Digital Equipment Corporation PDP- 
11/40 and 11/45 computers. It offers a number of features 
seldom found even in larger operating systems, including: 
(1) a hierarchical file system incorporating demountable 
volumes; (2) compatible file, device, and inter-process I/O; 
(3) the ability to initiate asynchronous processes; (4) sys¬ 
tem command language selectable on a per-user basis; and 
(5) over 100 subsystems including a dozen languages. 

— "The UNIX Timesharing System," 
D. Ritchie and K. Thompson, 1974 


The problems in this chapter are concerned with tools: version control systems, 
scripting languages, build systems, databases, and the networking stack. Such prob¬ 
lems are not commonly asked—expect them only if you are interviewing for a special¬ 
ized role, or if you claim specialized knowledge, e.g., network security or databases. 
We emphasize these are vast subjects, e.g., networking is taught as a sequence of 
courses in a university curriculum. Our goal here is to give you a flavor of what you 
might encounter. Adnan's Advanced Programming Tools course material, available 
freely online, contains lecture notes, homeworks, and labs that may serve as a good 
resource on the material in this chapter. 

Version control 

Version control systems are a cornerstone of software development—every developer 
should understand how to use them in an optimal way. 

24.1 Merging in a version control system 

What is merging in a version control system? Specifically, describe the limitations of 
line-based merging and ways in which they can be overcome. 

Solution: 

Modern version control systems allow developers to work concurrently on a per¬ 
sonal copy of the entire codebase. This allows software developers to work inde¬ 
pendently. The price for this merging: periodically, the personal copies need to be 
integrated to create a new shared version. There is the possibility that parallel changes 
are conflicting, and these must be resolved during the merge. 

By far, the most common approach to merging is text-based. Text-based merge 
tools view software as text. Line-based merging is a kind of text-based merging, 
specifically one in which each line is treated as an indivisible unit. The merging is 
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"three-way": the lowest common ancestor in the revision history is used in conjunc¬ 
tion with the two versions. With line-based merging of text files, common text lines 
can be detected in parallel modifications, as well as text lines that have been inserted, 
deleted, modified, or moved. Figure 24.1 on Page 425 shows an example of such a 
line-based merge. 

One issue with line-based merging is that it cannot handle two parallel modifica¬ 
tions to the same line: the two modifications cannot be combined, only one of the two 
modifications must be selected. A bigger issue is that a line-based merge may suc¬ 
ceed, but the resulting program is broken because of syntactic or semantic conflicts. 
In the scenario in Figure 24.1 on Page 425 the changes made by the two developers 
are made to independent lines, so the line-based merge shows no conflicts. However, 
the program will not compile because a function is called incorrectly. 

In spite of this, line-based merging is widely used because of its efficiency, scal- 
abilityand accuracy. A three-way, line-based merge tool in practice will merge 90% 
of the changed files without any issues. The challenge is to automate the remaining 
10% of the situations that cannot be merged automatically. 

Text-based merging fails because it does not consider any syntactic or semantic 
information. 

Syntactic merging takes the syntax of the programming language into account. 
Text-based merge often yields unimportant conflicts such as a code comment that has 
been changed by different developers or conflicts caused by code reformatting. A 
syntactic merge can ignore all these: it displays a merge conflict when the merged 
result is not syntactically correct. 

We illustrate syntactic merging with a small example: 

if (n%2 == Q) then m = n/2; 


Two developers change this code in a different ways, but with the same overall effect. 
The first updates it to 

if (n%2 == Q) then m = n/2 else m = (n-l)/2; 


and the second developer's update is 


m = n/2; 


Both changes to the same thing. However, a textual-merge will likely result in 


m := n div 2 else m := (n-l)/2 


which is syntactically incorrect. Syntactic merge identifies the conflict; it is the inte¬ 
grator's responsibility to manually resolve it, e.g., by removing the else portion. 

Syntactic merge tools are unable to detect some frequently occurring conflicts. 
For example, in the merge of Versions la and lb in Figure 24.1 on Page 425 will not 
compile, since the call to sum(10) has the wrong signature. Syntactic merge will 
not detect this conflict since the program is still syntactically correct. The conflict 
is a semantic conflict, specifically, a static semantic conflict, since it is detectable at 
compile-time. (The compile will return something like "function argument mismatch 
error".) 
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Technically, syntactic merge algorithms operate on the parse trees of the programs 
to be merged. More powerful static semantic merge algorithms have been proposed 
that operate on a graph representation of the programs, wherein definitions and usage 
are linked, making it easier to identify mismatched usage. 

Static semantic merging also has shortcomings. For example, suppose Point is 
a class representing 2D-points using Cartesian coordinates, and that this class has a 
distance function that returns ^jx 1 + y 1 . Suppose Alice checks out the project, and 
subclasses Point to create a class that supports polar coordinates, and uses Points's 
distance function to return the radius. Concurrently, Bob checks out the project and 
changes Point's distance function to return \x\ + \y\. Static semantic merging reports no 
conflicts: the merged program compiles without any trouble. However the behavior 
is not what was expected. 

Both syntactic merging and semantic merging greatly increase runtimes, and are 
very limiting since they are tightly coupled to specific programming languages. In 
practice, line-based merging is used in conjunction with a small set of unit tests (a 
"smoke suite") invoked with a pre-commit hook script. Compilation finds syntax 
and static semantics errors, and the tests (hopefully) identify deeper semantic issues. 


24.2 Hooks 

What are hooks in a version control system? Describe their applications. 

Solution: Version control systems such as git and SVN allow users to define actions 
to take around events such as commits, locking and unlocking files, and altering 
revision properties. These actions are specified as executable programs known as 
hook scripts. When the version control system gets to these events, it will check for 
the existence of a corresponding hook script, and execute it if it exists. 

Hook scripts have access to the action as it is taking place, and are passed cor¬ 
responding command-line arguments. For example, the pre-commit hook is passed 
the repository path and the transaction ID for the currently executing commit. If a 
hook script returns a nonzero exit code, the version control system aborts the action, 
returning the script's standard error output to the user. 

The following hook scripts are used most often: 

• pre-commit: Executed before a change is committed to the repository. Often 
used to check log messages, to format files, run a test suite, and to perform 
custom security or policy checking. 

• post-commit: Executed once the commit has completed. Often used to inform 
users about a completed commit, for example by sending an email to the team, 
or update the bug tracking system. 

As a rule, hooks should never alter the content of the transaction. At the very least, it 
can surprise a new developer; in the worst case, the change can result in buggy code. 


Scripting languages 

Scripting languages, such as AWK, Perl, and Python were originally developed as 
tools for quick hacks, rapid prototyping, and gluing together other programs. They 
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Figure 24.1: An example of a 3-way line based merge. 


have evolved into mainstream programming languages. Some of their characteristics 
are as follows. 

• Text strings are the basic (sometimes only) data type. 

• Associative arrays are a basic aggregate type. 

• Regexps are usually built in. 

• Programs are interpreted rather than compiled. 

• Declarations are often not required. 

They are easy to get started with, and can often do a great deal with very little code. 
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24.3 Is SCRIPTING MORE EFFICIENT? 


Your manager read a study in which a Python program was found to have taken one- 
tenth the development time then a program written in C++ for the same application, 
and now demands that you immediately program exclusively in Python. Tell him 
about five dangers of doing this. 

Solution: First, any case study is entirely anecdotal. It's affected by the proficiency of 
the developers, the nature of the program being written, etc. So no case study alone 
is sufficient as the basis for a language choice. 

Second, switching development environments for any team is an immediate slow¬ 
down, since the developers must learn all the new tricks and unlearn all the old ones. 
If a team wants to make this kind of switch, it needs to have a very long runway. 

Third, scripting languages are simply not good for some tasks. For example, many 
high-performance computing or systems tasks should not be done in Python. The 
lack of a type system means some serious bugs can go undetected. 

Fourth, choosing to program entirely in any language is a risky endeavor. Any 
decent engineering department should be willing to adopt the best tool for the job at 
hand. 

Finally, Python itself may be a risky choice at this point in time, since there is an 
ongoing shift from the 2.x branch to 3.x. If you choose to work in 2.x, you likely have 
a massive refactor into 3.x coming in the next few years; and if you choose to work 
in 3.x, you likely will run into libraries you simply cannot use because they don't yet 
support the latest versions of Python. 

24.4 Polymorphism with a scripting language 

Briefly, a Java interface is a kind of type specification. To be precise, it specifies a 
group of related methods with empty bodies. (All methods are nonstatic and public.) 
Interfaces have many benefits over, for example, abstract classes—in particular, they 
are a good way to implement polymorphic functions in Java. 

Describe the advantages and disadvantages of Python and Java for polymorphism. 
In particular, how would you implement a polymorphic function in Python? 
Solution: Java's polymorphism is first-class. The compiler is aware at compile-time 
of whatever interfaces (in the general sense) an object exposes—regardless of whether 
they derive from actual interfaces or from superclasses. As a result, a polymorphic 
function in Java has a guarantee that any argument to it implements the methods it 
will call during its execution. 

In Python, however, no such safety exists. So there are two tricks to use when 
implementing a polymorphic function in Python. First, you can specify your classes 
using multiple inheritance. In this case, many of the superclasses are simply interface 
specifications. These classes, mix-ins, may implement the desired methods or leave 
them for the subclass to implement. In effect, they provide a poor-man's interface. 
(However, they're not as robust as Java's solution since the method resolution order 
gets tricky in multiple-inheritance scenarios.) 

But there is a trickier problem here, which is that even if a class is implemented 
with a particular method, it's trivial at runtime to overwrite that method, e.g., with 
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None, before passing it as an argument. So you still don't have any guarantees. This 
motivates the second trick, which is the use of "duck typing"—that is, rather than 
trying to get some safety guarantee that cannot exist in Python, just inspect the object 
at runtime to see if it exposes the right interface. (The phrase comes from this adage: 
"If it looks like a duck, quacks like a duck, and acts like a duck, then it's probably 
a duck".) So, where in Java a developer might write a function that only accepts 
arguments that implement a Drawable interface, in Python you would simply call 
that object's draw method and hope for the best. Type checking still exists here; it's 
just deferred to runtime, which implies it may throw exceptions. The function is still 
quite polymorphic—more so, in some sense, since it can accept literally any object 
that has a draw method. 

Build systems 

A program typically cannot be executed immediately after change: intermediate steps 
are required to build the program and test it before deploying it. Other components 
can also be affected by changes, such as documentation generated from program text. 
Build systems automate these steps. 

24.5 Dependency analysis 

Argue that a build system such as Make which looks purely at the time stamps of 
source code and derived products when determining when a product needs to be 
rebuilt can end up spending more time building the product that is needed. How can 
the you avoid this? Are there any pitfalls to your approach? 

Solution: Suppose a source file is changed without its semantics being affected, e.g., 
a comment is added or it's reformatted. A build system like Make will propagate this 
change through dependencies, even though subsequent products are unchanged. 

One solution to this is as follows. During the reconstruction of a component A, a 
component called Anew is created and compared with the existing A. If Anew and 
A are different. Anew is copied to A. Otherwise, A remains unchanged, include the 
date that it was changed. 

This may not always be the reasonable thing to do. For example, even if the source 
files remain unchanged, environmental differences or changes to the compilation 
process may lead to the creation of different intermediate targets, which would ne¬ 
cessitate a full rebuild. Therefore each intermediate target must be rebuilt in order to 
compare against the existing components, but it's indeterminate before construction 
whether the new objects will be used or not. 

24.6 ANT vs. Maven 
Contrast ANT and Maven 

Solution: Historically, ANT was developed in the early days of Java to overcome the 
limitations of Make. It has tasks as a "primitive"—in Make these must be emulated 
using pseudo-targets, variables, etc. ANT is cross-platform—since Make largely 
scripts what you would do at the command-line, a Makefile written for Unix-like 
platforms will not work out-of-the-box on, say, Windows. 
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ANT itself has a number of shortcomings. These are overcome in Maven, as 
described below. 

• Maven is declarative whereas Ant is imperative in nature. You tell Maven what 
are the dependencies of your project—Maven will fetch those dependencies 
and build the artifact for you, e.g., a war file. In Ant, you have to provide all the 
dependencies of your project and tell Ant where to copy those dependencies. 

• With Maven, you can use "archetypes" to set up your project very quickly. 
For example you can quickly set up a Java Enterprise Edition project and get 
to coding in a matter of minutes. Maven knows what should be the project 
structure for this type of project and it will set it up for you. 

• Maven is excellent at dependency management. Maven automatically retrieves 
dependencies for your project. It will also retrieve any dependencies of depen¬ 
dencies. You never have to worry about providing the dependent libraries. Just 
declare what library and version you want and Maven will get it for you. 

• Maven has the concept of snapshot builds and release builds. During active 
development your project is built as a snapshot build. When you are ready to 
deploy the project, Maven will build a release version for you which can be put 
into a company wide Nexus repository. This way, all your release builds are in 
one repository that are accessible throughout the company. 

• Maven favors convention over configuration. In Ant, you have to declare a 
bunch of properties before you can get to work. Maven uses sensible defaults 
and if you follow the Maven recommended project layout, your build file is 
very small and everything works out of the box. 

• Maven has very good support for running automated tests. In fact, Maven will 
run your tests as part of the build process by default. You do not have to do 
anything special. 

• Maven provides hooks at different phases of the build and you can perform 
additional tasks as per your needs 

Database 

Most software systems today interact with databases, and it's important for develop¬ 
ers to have basic knowledge of databases. 

24.7 SQL vs. NoSQL 

Contrast SQL and NoSQL databases. 

Solution: A relational database is a set of tables (also known as relations). Each table 
consists of rows. A row is a set of columns (also known as fields or attributes), which 
could be of various types (integer, float, date, fixed-size string, variable-size string, 
etc.). SQL is the language used to create and manipulate such databases. MySQL is 
a popular relational database. 

A NoSQL database provides a mechanism for storage and retrieval of data which 
is modeled by means other than the tabular relations used in relational databases. 
MongoDB is a popular NoSQL database. The analog of a table in MongoDB is 
a collection, which consists of documents. Collections do not enforce a schema. 
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Documents can be viewed as hash maps from fields to values. Documents within a 
collection can have different fields. 

A key benefit of NoSQL databases is simplicity of design: a field can trivially be 
added to new documents in a collection without this affecting documents already in 
the database. This makes NoSQL a popular choice for startups which have an agile 
development environment. 

The data structures used by NoSQL databases, e.g., key-value pairs in MongoDB, 
are different from those used by default in relational databases, making some opera¬ 
tions faster in NoSQL. NoSQL databases are typically much simpler to scale horizon¬ 
tally, i.e., to spread documents from a collection across a cluster of machines. 

A key benefit of relational databases include support for ACID transactions. ACID 
(Atomicity, Consistency, Isolation, Durability) is a set of properties that guarantee 
that database transactions are processed reliably. For example, there is no way in 
MongoDB to remove an item from inventory and add it to a customer's order as an 
atomic operation. 

Another benefit of relational database is that their query language, SQL, is declar¬ 
ative, and relatively simple. In contrast, complex queries in a NoSQL database have 
to be implemented programmatically. (It's often the case that the NoSQL database 
will have some support for translating SQL into the equivalent NoSQL program.) 


24.8 Normalization 

What is database normalization? What are its advantages and disadvantages? 
Solution: Database normalization is the process of organizing the columns and tables 
of a relational database to minimize data redundancy. Specifically, normalization 
involves decomposing a table into less redundant tables without losing information, 
thereby enforcing integrity and saving space. 

The central idea is the use of "foreign keys". A foreign key is a field (or collection 
of fields) in one table that uniquely identifies a row of another table. In simpler 
words, the foreign key is defined in a second table, but it refers to the unique key in 
the first table. For example, a table called Employee may use a unique key called em- 
ployee_id. Another table called Employee Details has a foreign key which references 
employee_id in order to uniquely identify the relationship between both the tables. 
The objective is to isolate data so that additions, deletions, and modifications of an 
attribute can be made in just one table and then propagated through the rest of the 
database using the defined foreign keys. 

The primary drawback of normalization is performance—often, a large number of 
joins are required to recover the records the application needs to function. 


24.9 SQL design 

Write SQL commands to create a table appropriate for representing students as well 
as SQL commands that illustrate how to add, delete, and update students, as well as 
search for students by GPA with results sorted by GPA. 
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Then add a table of faculty advisors, and describe how to model adding an advisor 
to each student. Write a SQL query that returns the students who have an advisor 
with a specific name. 

Solution: Natural fields for a student are name, birthday, GPA, and gradu¬ 
ating year. In addition, we would like a unique key to serve as an iden¬ 
tifier. A SQL statement to create a student table would look like the fol¬ 
lowing: CREATE TABLE students (id INT AUTO_INCREMENT, name VARCHAR(3(S>) , 
birthday DATE, gpa FLOAT, graduate.year INT, PRIMARY KEY(id)); . Here are 
examples of SQL statements for updating the students table. 

• Add a student 

- INSERT INTO students(name, birthday, gpa, graduate_year) 

VALUES (’Cantor’, ’1987-10-22’, 3.9, 2009); 

• Delete students who graduated before 1985. 

- DELETE FROM students WHERE graduate_year < 1985; 

• Update the GPA and graduating year of the student with id 28 to 3.14 and 2015, 
respectively. 

- UPDATE students SET gpa = 3.14, graduate_year = 2015 WHERE id 
= 28; 

The following query returns students with a GPA greater than 3.5, sorted by GPA. 
It includes each student's name, GPA, and graduating year. SELECT gpa, name, 
graduate_year FROM students WHERE gpa >3.5 ORDER BY gpa DESC; 

Now we turn to the second part of the problem. Here is a sample table for faculty 
advisers. 


id 

name 

title 

1 

Church 

Dean 

2 

Tarski 

Professor 


We add a new column advisor_id to the students table. This is the foreign key on 
the previous page. As an example, here's how to return students with GPA whose 
advisor is Church: SELECT s.name, s.gpa FROM students s, advisors p WHERE 
s.advisor_id = p.id AND p.name = ’Church’;. 

Variant: What is a SQL join? Suppose you have a table of courses and a table of 
students. How might a SQL join arise naturally in such a database? 

Networking 

Most software systems today interact with the network, and it's important for de¬ 
velopers even higher up the stack to have basic knowledge of networking. Here are 
networking questions you may encounter in an interview setting. 

24.10 IP, TCP, and HTTP 

Explain what IP, TCP, and HTTP are. Emphasize the differences between them. 
Solution: IP, TCP, and HTTP are networking protocols—they help move information 
between two hosts on a network. 

IP, the Internet Protocol, is the lowest-level protocol of the three. It is independent 
of the physical channel (unlike for example, WiFi or wired Ethernet). It's primary 
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focus is on getting individual packets from one host to another. Each IP packet has a 
header that include the source and destination IP address, the length of the packet, the 
type of the protocol it's layering (TCP and UDP are common), and an error checking 
code. 

IP moves packets through a network consisting of links and routers. A router 
has multiple input and output ports. It forwards packets through the network using 
routing tables that tell the router which output port to forward in incoming packet to. 
Specifically, a routing table maps IP prefixes to output ports. For example, an entry in 
the routing table may indicate that all packets with destination IP address matching 
171.23.*.* are to be forwarded out the interface port with id 17. IP routers exchange 
"link state information" to compute the routing tables; this computation is a kind of 
shortest path algorithm. 

TCP, the Transmission Control Protocol, is an end-to-end protocol built on top of IP. 
It creates a a continuous reliable bidirectional connection. It does this by maintaining 
state at the two hosts—this state includes IP packets that have been received, as 
well as sending acknowledgements. It responds to dropped packets by requesting 
retransmits—each TCP packet has a sequence number. The receiver signals the 
transmitter to reduce its transmission rate if the drop rate becomes high—it does 
this by reducing the received window size. TCP also "demultiplexes" incoming data 
using the concept of port number—this allows multiple applications on a host with a 
single IP address to work simultaneously. 

HTTP, the Hyper Text Transfer Protocol, is built on top of TCP. An HTTP request or 
response can be viewed as a text string consisting of a header and a body. The HTTP 
request and responses header specify the type of the data through the Content-Type 
field. The response header includes a code, which could indicate success ("200"), a 
variety of failures ("404"—resource not found), or more specialized scenarios ("302"— 
use cached version). A few other protocol fields are cookies, request size, compression 
used. 

HTTP was designed to enable the world-wide web, and focused on delivering web 
pages in a single logical transaction. It's generality means that it has grown to provide 
much more, e.g., it's commonly used to access remote procedures ("services"). An 
object—which could be a web page, a file, or a service—are addressed using a URL, 
Uniform Resource Locator, which is a domain name, a path, and optional arguments. 
When used to implement services, a common paradigm is for the client to send a 
request using URL arguments, a JSON object, and files to be uploaded, and the server 
responds with JSON. 

24.11 HTTPS 

Describe HTTPS operationally, as well as the ideas underlying it. 

Solution: In a nutshell, HTTPS is HTTP with SSL encryption. Because of the com¬ 
mercial importance of the Internet, and the prevalence of wireless networks, it's 
imperative to protect data from eavesdroppers—this is not done automatically by 
IP/TCP/HTTP. 

Conceptually, one way to encrypt a channel is for the communicating parties to 
XOR each transmitted byte with a secret byte—the receiver then XORs the received 
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bytes with the secret. This scheme is staggeringly weak—an eavesdropper can look 
at statistics of the transmitted data to easily figure out which of the 256 bytes is the 
secret. It leaves open the question of how to get the secrete byte to the receiver. 
Furthermore, a client who initiates a transaction has no way of knowing if someone 
has hijacked the network and directed traffic from the client to a malicious server. 

SSL resolves these issues. First of all, it uses public key cryptography for key 
exchange—the idea is that the public key is used by the client to encrypt and send 
a secret key (which can be viewed as a generalization of the secret byte; 256 bits is 
the common size) to the server. This secret key is used to scramble the data to be 
transmitted in a recoverable way. The secret key is generated via hashing a random 
number. Public key cryptography is based on the fact that it's possible to create a 
one-way function, i.e., a function that can be made public and is easy to compute in 
one direction, but hard to invert with the public information about it. Rivest, Shamir, 
and Adleman introduced a prototypical such function which is based on several 
number theoretic facts, such as the relative ease with which primes can be created, 
that computing a b mod c is fast, and that factorization is (likely) hard. 

Private key cryptography involves a single key which is a shared secret between 
the sender and recipient. Private key cryptography offers many orders of magnitude 
higher bandwidth than public key cryptography. For this reason, once the secret 
key has been exchanged via public key cryptography, HTTPS turns to private key 
cryptography, specifically AES. AES operates on 128 bit blocks of data. (If the data 
to be transmitted is less than 128 bits, it's padded up; if it's greater than 128 bits, it's 
broken down into 128 bit chunks.) The 128 bit block is represented as 16 bytes in a 
4x4 matrix. The secret key is used to select an ordering in which to permute the 
bytes; it is also used to XOR the bits. This operation is reversible using the secret key. 

A small shortcoming of the scheme described above is that it is susceptible to 
replay attacks—an adversary may not know the exact contents of a transmission, but 
may know that it represent an action ("add 10 gold coins to my World-Of-Warcraft 
account"). This can be handled by having the server send a random number which 
must accompany client requests. 

HTTPS adds another layer of security—SSL certificates. The third party issuing 
the SSL certificate verifies that the HTTPS server is on the domain that it claims to be 
on, which precludes someone who has hijacked the network from pretending to be 
the server being requested by the client. This check is orthogonal to the essence of 
SSL (private key exchange and AES), and it can be disabled, e.g., for testing a server 
locally. 

24.12 DNS 

Describe DNS, including both operational as well as implementation aspects. 
Solution: DNS is the system which translates domain names—www.google.com— 
to numerical IP addresses—216.58.218.110. It is essential to the functioning of the 
Internet. DNS is what makes it possible for a browser to access a website—the routers 
in the Internet network rely on IP addresses to lookup next-hops for packets. 

There are many challenges to implementing DNS. Here are some of the most 
important ones. 
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• There are hundreds of millions of domains. 

• There are hundreds of billions of DNS lookups a day. 

• Domains and IP addresses are added and updated continuously. 

To solve these problems, DNS is implemented as a distributed directory service. 
A DNS lookup is addressed to a DNS server. Each DNS server stores a database 
of domain names to IP addresses; if it cannot find a domain name being queried in 
this database it forwards the request to other DNS servers. These requests can get 
forwarded all the way up to the root name servers, which are responsible for top-level 
domains. 

At the time of writing there are 13 root name servers. This does not mean there 
are exactly 13 physical servers. Each operator uses redundancy to provide reliable 
service. Additionally, these servers are replicated across multiple locations. Key to 
performance is the heavy use of caching, in the client itself, as well as in the DNS 
servers. 
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Part IV 

The Honors Class 



Chapter 


Honors Class 


The supply of problems in mathematics is inex¬ 
haustible, and as soon as one problem is solved 
numerous others come forth in its place. 

— "Mathematical Problems," 
D. Hilbert, 1900 


This chapter contains problems that are more difficult to solve than the ones presented 
earlier. Many of them are commonly asked at interviews, albeit with the expectation 
that the candidate will not deliver the best solution. 

There are several reasons why we included these problems: 

• Although mastering these problems is not essential to your success, if you do 
solve them, you will have put yourself into a very strong position, similar to 
training for a race with ankle weights. 

• If you are asked one of these questions or a variant thereof, and you successfully 
derive the best solution, you will have strongly impressed your interviewer. The 
implications of this go beyond being made an offer—your salary, position, and 
status will all go up. 

• Some of the problems are appropriate for candidates interviewing for special¬ 
ized positions, e.g., optimizing large scale distributed systems, machine learn¬ 
ing for computational finance, etc. They are also commonly asked of candidates 
with advanced degrees. 

• Finally, if you love a good challenge, these problems will provide it! 

You should be happy if your interviewer asks you a hard question—it implies high 
expectations, and is an opportunity to shine, compared, for example, to being asked 
to write a program that tests if a string is palindromic. 

Problems roughly follow the sequence of topics of the previous chapters, i.e., we 
begin with primitive types, and end with graphs. We recommend you solve these 
problems in a randomized order. The following problems are a good place to begin 
with: Problems 25.1,25.6,25.8,25.9,25.12,25.13,25.19,25.22,25.25,25.26,25.33. White 
ninja (©*) problems are though challenging, should be solvable by a good candidate 
given enough time. Black ninja (®<) problems are exceptionally difficult, and are 
suitable for testing a candidate's response to stress, as described on Pagel8. 

25.1 Compute the greatest common divisor & 

The greatest common divisor (GCD) of positive integers x and y is the largest integer d 
such that d divides x evenly, and d divides y evenly, i.e., x mod d- 0 and y mod d — 0. 
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Design an efficient algorithm for computing the GCD of two numbers without using 
multiplication, division or the modulus operators. 

Hint: Use case analysis: both even; both odd; one even and one odd. 

Solution: The straightforward algorithm is based on recursion. If x = y, GCD(x, y) = 
x; otherwise, assume without loss of generality, that x > y. Then GCD(x, y) is the 
GCD (x-y,y). 

The recursive algorithm based on the above does not use multiplication, division 
or modulus, but for some inputs it is very slow. As an example, if the input is x = 2 n , 
y = 2, the algorithm makes 2” _1 recursive calls. The time complexity can be improved 
by observing that the repeated subtraction amounts to division, i.e., when x > y, 
GCD(x, y) = GCD(y, x mod y), but this approach uses integer division which was 
explicitly disallowed in the problem statement. 

Here is a fast solution, which is also based on recursion, but does not use general 
multiplication or division. Instead it special-cases division to division by 2. 

An example is illustrative. Suppose we were to compute the GCD of 24 and 300. 
Instead of repeatedly subtracting 24 from 300, we can observe that since both are 
even, the result is 2 X GCD(12,150). Dividing by 2 is a right shift by 1, so we do not 
need a general division operation. Since 12 and 150 are both even, GCD(12,150) = 
2 x GCD(6,75). Since 75 is odd, the GCD of 6 and 75 is the same as the GCD of 3 
and 75, since 2 cannot divide 75. The GCD of 3 and 75 is the GCD of 3 and 75 - 3 = 
72. Repeatedly applying the same logic, GCD(3,72) = GCD(3,36) = GCD(3,18) = 
GCD(3, 9) = GCD(3,6) = GCD(3,3) = 3. This implies GCD(24,300) = 2 x 2 X 3 = 12. 

More generally, the base case is when the two arguments are equal. Otherwise, we 
check if none, one, or both numbers are even. If both are even, we compute the GCD 
of the halves of the original numbers, and return that result times 2; if exactly one is 
even, we half it, and return the GCD of the resulting pair; if both are odd, we subtract 
the smaller from the larger and return the GCD of the resulting pair. Multiplication 
by 2 is trivially implemented with a single left shift. Division by 2 is done with a 
single right shift. 

public static long GCD (long x, long y) { 
if (x == y) { 
return x; 

} else if C(x & 1) == ® && (y & 1) == ®) { // x and y are even. 

return GCD(x »> 1, y »> 1) « 1; 

} else if ((x <& 1) == ® &<& (y <& 1) != ®) { // x is even, y is odd. 
return GCD(x »> 1, y) ; 

} else if ((x & 1) != ® &<& (y <& 1) == ®) { // x is odd, y is even. 

return GCD(x, y »> 1); 

} else if (x > y) { // Both x and y are odd, and x > y. 
return GCD(x - y, y) ; 

} 

return GCD(x, y - x); // Both x and y are odd, and x <= y. 

} 


Note that the last step leads to a recursive call with one even and one odd number. 
Consequently, in every two calls, we reduce the combined bit length of the two 
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numbers by at least one, meaning that the time complexity is proportional to the sum 
of the number of bits in x and y, i.e., 0( log* + log y)). 


25.2 Find the first missing positive entry ©* 

Let A be an array of length n. Design an algorithm to find the smallest positive integer 
which is not present in A. You do not need to preserve the contents of A. For example, 
if A = (3,5,4, -1,5,1, -1), the smallest positive integer not present in A is 2. 

Hint: First, find an upper bound for x. 

Solution: A brute-force approach is to sort A and iterate through it looking for the 
first gap in the entries after we see an entry equal to 0. The time complexity is that of 
sorting, i.e., 0(n log n). 

Since all we want is the smallest positive number in A , we explore other algo¬ 
rithms that do not rely on sorting. We could store the entries in A in a hash table S 
(Chapter 13), and then iterate through the positive integers 1,2,3,... looking for the 
first one that is not in S. The time complexity is 0(n) to create S, and then 0(n) to 
perform the lookups, since we must find a missing entry by the time we get to n +1 as 
there are only n entries. Therefore the time complexity is 0(n). The space complexity 
is 0(n), e.g., if the entries from A are all distinct positive integers. 

The problem statement gives us a hint which we can use to reduce the space 
complexity. Instead of using an external hash table to store the set of positive integers, 
we can use A itself. Specifically, if A contains k between 1 and n, we set A[k- 1] to 
k. (We use k - 1 because we need to use all n entries, including the entry at index 
0, which will be used to record the presence of 1.) Note that we need to save the 
presence of the existing entry in A [k -1] if it is between 1 and n. Because A contains n 
entries, the smallest positive number that is missing in A cannot be greater than n + 1. 

For example, let A = (3,4,0,2), n = 4. we begin by recording the presence of 3 
by writing it in A[3 - 1]; we save the current entry at index 2 by writing it to A[0]. 
Now A = (0,4,3,2). Since 0 is outside the range of interest, we advance to A[l], i.e., 
4, which is within the range of interest. We write 4 in A[4 - 1], and save the value at 
that location to index 1, and A becomes (0,2,3,4). The value at A[l] already indicates 
that a 2 is present, so we advance. The same holds for A[2] and A[3]. 

Now we make a pass through A looking for the first index i such that A[i\ ± i + 1; 
this is the smallest missing positive entry, which is 1 for our example. 

public static int findFirstMissingPositive(List<Integer> A) { 

// Record which values are present by writing A.get(i) to index A.get(i) - 1 
// if A.get(i) is between 1 and A.size(), inclusive. We save the value at 
// index A.get(i) - 1 by swapping it with the entry at i. If A.get(i) is 
// negative or greater than n, we just advance i. 
int i = Q; 

while (i < A.sizeO) { 

if (A. get (i) > ® &<& A.get(i) <= A.sizeO 
&& A.get(A.get(i) - 1) != A.get(i)) { 

Collections.swap(A, i, A.get(i) - 1); 

} else { 

++i ; 
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} 

} 

// Second pass through A to search for the first index i such that A.get(i) 
// != i + 1, indicating that i + 1 is absent. If all numbers between 1 and 
// A.size() are present, the smallest missing positive is A.size() + 1. 
for (i = Q; i < A.sizeO; ++i) { 
if (A.get(i) != i + 1) { 
return i + 1; 

} 

} 

return A.sizeO + 1; 


The time complexity is 0(ri), since we perform a constant amount of work per entry. 
Because we reuse A, the space complexity is 0(1). 


25.3 Buy and sell a stock k times ®< 

This problem generalizes the buy and sell problem introduced on Page 1. 

Write a program to compute the maximum profit that can be made by buying and 
selling a share k times over a given day range. Your program takes k and an array of 
daily stock prices as input. 

Solution: Here is a straightforward algorithm. Iterate over j from 1 to k and iterate 
through A, recording for each index i the best solution for A[0 : i] with j pairs. We 
store these solutions in an auxiliary array of length n. The overall time complexity 
will be 0(kn 2 y, by reusing the arrays, we can reduce the additional space complexity 
to 0(n). 

We can improve the time complexity to 0(kn), and the additional space complexity 
to 0(k) as follows. Define Bl to be the most money you can have if you must make 
j — 1 buy-sell transactions prior to i and buy at i. Define S ] . to be the maximum profit 
achievable with j buys and sells with the ;th sell taking place at i. Then the following 
mutual recurrence holds: 

Si = A[i\ +max Bl, 

1 i'<i 1 

Bl = rnaxSC 1 -^] 

1 i'<i 1 

The key to achieving an 0(kn) time bound is the observation that computing B 
and S requires computing max,'< ; Bf 1 and max,/<j sj -1 . These two quantities can be 
computed in constant time for each i and j with a conditional update. In code: 

public static double maxKPairsProfits(List<Double> A, int k) { 

List<Double> kSum = new ArrayList<>(); 
for (int i = ®; i<k* 2; ++i) { 
kSum.add(Double.NEGATIVE_INFINITY); 

} 

for (int i = ®; i < A.sizeO; ++i) { 

List<Double> preKSum = new ArrayListo(kSum) ; 
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for (int j = ®, sign = -1; j < kSum.size() && j <= i; ++j, sign *= -1) { 
double diff = sign * A.get(i) + (j == ® ? ® : preKSum.get(j - 1)); 
kSum.set(j, Math.max(diff, preKSum.get(j))); 

} 

} 

// Returns the last selling profits as the answer. 
return kSum.get(kSum.size() - 1); 


Variant: Write a program that determines the maximum profit that can be obtained 
when you can buy and sell a single share an unlimited number of times, subject to 
the constraint that a buy must occur more than one day after the previous sell. 

25.4 Compute the maximum product of all entries but one 

Suppose you are given an arrays of integers, and are asked to find the largest product 
that can be made by multiplying all but one of the entries in A. (You cannot use an 
entry more than once.) For example, if A = (3, 2,5, 4), the result is 3 x 5 x 4 = 60, if 
A = (3, 2, —1,4), the result is 3 X 2 X 4 = 24, and if A = (3, 2, —1,4, — 1,6), the result is 
3 x -1 x 4 x -1 x 6 = 72. 

One approach is to form the product P of all the elements, and then find the 
maximum of P/A[i] over all i. This takes n - 1 multiplications (to form P) and n 
divisions (to compute each P/A[i]). Suppose because of finite precision considerations 
we cannot use a division-based approach; we can only use multiplications. The brute- 
force solution entails computing all n products of n - 1 elements; each such product 
takes n-2 multiplications, i.e., 0(n 2 ) time complexity. 

Given an array A of length n whose entries are integers, compute the largest product 
that can be made using n — 1 entries in A. You cannot use an entry more than once. 
Array entries may be positive, negative, or 0. Your algorithm cannot use the division 
operator, explicitly or implicitly. 

Hint: Consider the products of the first i - 1 and the last n — i elements. Alternatively, count the 
number of negative entries and zero entries. 

Solution: The brute-force approach to compute P/A[i\ is to multiplying the entries 
appearing before i with those that appear after i. This leads to n(n - 2) multiplications, 
since for each term P/A[i\ we need n-2 multiplications. 

Note that there is substantial overlap in computation when computing P/A[i\ and 
P/A[i +1]. In particular, the product of the entries appearing before i + 1 is A[i\ 
times the product of the entries appearing before i. We can compute all products of 
entries before i with n — 1 multiplications, and all products of entries after i with n — 1 
multiplications, and store these values in an array of prefix products, and an array 
of suffix products. The desired result then is the maximum prefix-suffix product. 
A slightly more space efficient approach is to build the suffix product array and 
then iterate forward through the array, using a single variable to compute the prefix 
products. This prefix product is multiplied with the corresponding suffix product 
and the maximum prefix-suffix product is updated if required. 
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public static int findBiggestProductNMinusOneProduct(Listdnteger> A) { 
// Builds suffix products. 
int product = 1; 

List dnteger > suf f ixProducts 

= new ArrayList<>(Collections.nCopies(A.size(), ®)); 
for (int i = A.sizeO - 1; i >= ®; --i) { 
product *= A.get(i); 
suffixProducts.set(i, product); 

} 

// Finds the biggest product of (n - 1) numbers. 

int prefixProduct = 1; 

int maxProduct = Integer.MIN_VALUE; 

for (int i = ®; i < A.sizeO; ++i) { 

int suf f ixProduct = i + 1 < A.sizeO ? suff ixProducts . get (i + 1) : 1; 

maxProduct = Math.max(maxProduct, prefixProduct * suffixProduct); 
prefixProduct *= A.get(i); 

} 

return maxProduct; 


The time complexity is 0(n); the space complexity is 0(n), since the solution uses a 
single array of length n. 

We now solve this problem in 0(n) time and 0(1) additional storage. The insight 
comes from the fact that if there are no negative entries, the maximum product comes 
from using all but the smallest element. (Note that this result is correct if the number 
of 0 entries is zero, one, or more.) 

If the number of negative entries is odd, regardless of how many 0 entries and 
positive entries, the maximum product uses all entries except for the negative entry 
with the smallest absolute value, i.e., the greatest negative entry. 

Going further, if the number of negative entries is even, the maximum product 
again uses all but the smallest nonnegative element, assuming the number of nonneg¬ 
ative entries is greater than zero. (This is correct, even in the presence of 0 entries.) 

If the number of negative entries is even, and there are no nonnegative entries, the 
result must be negative. Since we want the largest product, we leave out the entry 
whose magnitude is largest, i.e., the least negative entry. 

This analysis yields a two-stage algorithm. First, determine the applicable sce¬ 
nario, e.g., are there an even number of negative entries? Consequently, perform the 
actual multiplication to get the result. 

public static int findBiggestNMinusOneProduct(List<Integer> A) { 
int leastNonnegativeldx = -1; 

int numberOfNegatives = ®, greatestNegativeldx = -1, leastNegativeldx = -1; 

// Identify the least negative , greatest negative , and least nonnegative 
// entries. 

for (int i = ®; i < A.sizeO; ++i) { 
if (A.get (i) < ®) { 

++numberOfNegatives ; 

if (leastNegativeldx == -1 || A.get(leastNegativeldx) < A.get(i)) { 
leastNegativeldx = i; 

} 
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if (greatestNegativeldx == -1 

|| A.get(i) < A.get(greatestNegativeldx)) { 
greatestNegativeldx = i; 

} 

} else if (A.get(i) >= ®) { 
if (leastNonnegativeldx == -1 

|| A.get(i) < A.get(leastNonnegativeldx)) { 
leastNonnegativeldx = i; 

} 

} 

} 

int product = 1; 

int IdxToSkip = (numberOfNegatives % 2) != ® 

? leastNegativeldx 

// Check if there are any nonnegative entry. 

: (leastNonnegativeldx != -1 ? leastNonnegativeldx 

: greatestNegativeldx); 

for (int i = ®; i < A.sizeO; ++i) { 
if (i != IdxToSkip) { 
product *= A.get(i); 

} 

} 

return product; 


The algorithm performs a traversal of the array, with a constant amount of com¬ 
putation per entry, a nested conditional, followed by another traversal of the array, 
with a constant amount of computation per entry. Hence, the time complexity is 
0(n) + 0(1) + 0(n) = 0(n). The additional space complexity is 0(1), corresponding to 
the local variables. 

Variant: Let A be as above. Compute an array B where B[i] is the product of all 
elements in A except A[i\. You cannot use division. Your time complexity should be 
0(n ), and you can only use 0(1) additional space. 

Variant: Let A be as above. Compute the maximum over the product of all triples of 
distinct elements of A. 


25.5 Compute the longest contiguous increasing subarray & 

An array is increasing if each element is less than its succeeding element except for 
the last element. 

Implement an algorithm that takes as input an array A of n elements, and returns the 
beginning and ending indices of a longest increasing subarray of A. For example, if 
A = (2,11,3,5,13,7,19,17,23), the longest increasing subarray is (3,5,13), and you 
should return (2,4). 

Hint: If A[t] > A[i + 1], instead of checking A[i + 1] < A[i + 2], go further out in the array. 

Solution: The brute-force algorithm is to test every subarray: two nested loops 
iterate through the starting and ending index of the subarray, an inner loop to test 
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if the subarray is increasing, and some logic to track the longest subarray. The time 
complexity is 0(n 3 ), e.g., for the array (0,1,2,3 ,n - 1). The time complexity can 
easily be improved to 0(n 2 ) by caching whether the subarray A[i : j] is increasing 
when examining A[i : j + 1]. 

Looking more carefully at the example, the longest subarray ending at 3 is A [2 : 2], 
because 3 < 11. Conversely, since 13 > 5, the longest subarray ending at 13 is the 
longest subarray ending at 5 (which is A[2 : 3]) together with 13, i.e., A[ 2:4]. In 
general, the longest increasing subarray ending at index j + 1 inclusive is 

1. the single entry A[j + 1], if A[j + 1] < A[j], or 

2. the longest increasing subarray ending at index j together with A[j + 1], if 
A[j + l]>A[j]. 

This fact can be used directly to improve the brute-force algorithm to one whose time 
complexity is 0(n). 

The additional space complexity is 0(1), since all we need is the length of the 
longest subarray ending at j when processing j + 1. Two additional variables can be 
used to hold the length and ending index of the longest increasing subarray seen so 
far. 

We can heuristically improve upon the 0(n) algorithm by observing that if A [i -1 ] it 
A[i] (i.e., we are starting to look for a new subarray starting at i) and the longest 
contiguous subarray seen up to index i has length L, we can move on to index i + L 
and work backwards towards i. Specifically, if for any j,i < j < i + L we have 
A[j - 1] it A[j], we can skip the earlier indices. For example, after processing 13, we 
work our way back from the entry at index 4 + 3 = 7, i.e., 13's index plus the length 
of the longest increasing subarray seen so far (3). Since A[7] = 17 < A[6] = 19, we do 
not need to examine prior entries—no increasing array ending at A[7] can be longer 
than the current best. 


// Represent subarray by starting and ending indices, inclusive. 

private static class Subarray { 
public Integer start; 
public Integer end; 

public Subarray(Integer start, Integer end) { 
this. start = start; 
this.end = end; 

} 

} 

public static Subarray findLongestlncreasingSubarray(Listdnteger> A) { 
int maxLength = 1; 

Subarray ans = new Subarray(®, ®) ; 
int i = ®; 

while (i < A.size() - maxLength) { 

// Backward check and skip if A[j - 1] >= A[jJ. 
boolean isSkippable = false; 
for (int j = i + maxLength; j > i; --j) { 
if (A.get(j - 1) >= A.get(j)) { 

i = j ; 

isSkippable = true; 

break; 

} 
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// Forward check if it is not skippable. 
if (!isSkippable) { 
i += maxLength; 

while (i < A.size() && A.get(i - 1) < A.get(i)) { 
++i ; 

++maxLength; 

} 

ans = new Subarray(i - maxLength, i - 1); 

} 

} 

return ans; 


Skipping is a heuristic in that it does not improve the worst-case complexity. If the 
array consists of alternating Os and Is, we still examine each element, implying an 
0(n) time bound, but the best-case complexity reduces to <9(max(n/L,L)), where L is 
the length of the longest increasing subarray. 


25.6 Rotate an array &■ 

Let A be an array of n elements. If memory is not a concern, rotating A by i positions 
is trivial; we create a new array B of length n, and set B[j] = A[(i + j) mod n] for each j. 
If all we have is additional storage for c elements, we can repeatedly rotate the array 
by c a total of \i/c\ times; this increases the time complexity to 0(n\i/c ]). 

Design an algorithm for rotating an array A of n elements to the right by i positions. 
Do not use library functions implementing rotate. 

Hint: Use concrete examples to form a hypothesis relating n, i, and the number of cycles. 

Solution: There are two brute-force algorithms: perform shift-by-one i times, which 
has 0(ni) time complexity and <9(1) space complexity. The other is to use an additional 
array of length i as a buffer to move elements i at a time. This has <9(n) time complexity 
and <9(0 space complexity. 

The key to achieving both 0(n) time complexity and <9(1) space complexity is 
to use the fact that a permutation can be applied using constant additional storage 
(Problem 6.9 on Page 74) with the permutation corresponding to a rotation. 

A rotation by itself is not a cyclic permutation. However, a rotation is a per¬ 
mutation, and as such can be decomposed to a set of cyclic permutations. For 
example, for the case where n - 6 and i = 2, the rotation corresponds to the permu¬ 
tation (4,5,1,2,3,4). This permutation can be achieved by the cyclic permutations 
(0,2,4) and (1,3,5). Similarly, when n - 15 and i = 6, the cycles are (0,6,12,3,9), 
(1,7,13,4,10), and (2,8,14,5,11). 

The examples lead us to conjecture the following: 

(1.) All cycles have the same length, and are a shifted version of the cycle (0, i mod 
n, 2 i mod n, ..., (Z - 1 )i mod n). 

(2.) The number of cycles is the GCD of n and i. 
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These conjectures can be justified on heuristic grounds. (A formal proof requires 
looking at the prime factorizations for i and n.) 

Assuming these conjectures to be correct, we can apply the rotation one cycle at a 
time, as follows. The first elements of the different cyclic permutations are at indices 
0,1,2,..., GCD(n, i) - 1. For each cycle, we apply it by shifting elements in the cycle 
one-at-a-time. 


public static void rotateArray (int rotateAmount, Listdnteger> A) { 
rotateAmount %= A.sizeO; 
if (rotateAmount == ®) { 
return; 

} 

int numCycles = (int )GCD1.GCD(A.size(), rotateAmount); 
int cycleLength = A.sizeO / numCycles; 

for (int c = ®; c < numCycles; ++c) { 

applyCyclicPermutation(rotateAmount, c, cycleLength, A); 

} 


private static void applyCyclicPermutation(int rotateAmount, int offset, 

int cycleLength, List dnteger > A) { 

int temp = A.get(offset); 

for (int i = 1; i < cycleLength; ++i) { 

int val = A. get ((offset + i * rotateAmount) % A.sizeO); 

A. set ((of fset + i * rotateAmount) % A.sizeO, temp); 
temp = val ; 

} 

A.set(offset, temp); 


The time complexity is 0{ri), since we perform a constant amount of work per entry. 
The space complexity is (9(1). 

We now provide an alternative to the permutation approach. The new solution 
works well in practice and is considerably simpler. Assume that A - (1,2,3,4,0,17), 
and i = 2. Then in the rotated A there are two subarrays, (1,2,3,4) and {a,b) that 
keep their original orders. Therefore, rotation can be seen as the exchanges of the 
two subarrays of A. It is easy to perform these exchanges in 0(n) time. To implement 
these exchanges to use 0(1) space we use an array-reverse function. Using A and i as 
an example, we first reverse A to get A ' ((1,2,3,4,0,17) becomes (1?, 0,4,3,2,1)), then 
reverse the first i elements of A' ((17,0,4,3,2,1) becomes (0,1?,4,3,2,1)), and reverse 
the remaining elements starting from the zth element of A' ((0,17,4,3,2,1) becomes 
(a, b, 1,2,3,4)) which yields the rotated A. 

public static void rotateArray (int i, List<Integer> A) { 
i %= A . size() ; 
reverse(®, A.sizeO, A); 
reverse(®, i, A) ; 
reverse(i, A.sizeO, A); 

} 

private static void reverse (int begin, int end, Listdnteger> A) { 
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for (int i = begin, j = end - 1; i < j; ++i, --j) { 
Collections.swap(A, i, j); 

} 


We note in passing that a completely different approach is to perform the rotation 
in blocks of k, reusing freed space for temporary storage. It can be made to work, 
but the final rotation has to be performed very carefully, and the resulting code is 
complex. 


25.7 Identify positions attacked by rooks 

This problem is concerned with computing squares in a chessboard which can be at¬ 
tacked by rooks that have been placed at arbitrary locations. The scenario is illustrated 
in Figure 25.1(a). 



(a) Initial placement of 5 rooks on 
an 8 x 8 chessboard. 
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(b) Rook placement from (a) en¬ 
coded using an 8 x 8 2D array—a 
0 indicates a rook is placed at that 
position. 

Figure 25.1: Rook attack. 
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(c) 2D array encoding positions 
that can be attacked by rooks 
placed as in (a)—a 0 indicates an 
attacked position. 


Write a program which takes as input a 2D array A of Is and Os, where the Os encode 
the positions of rooks on an n X m chessboard, as show in Figure 25.1(b) and updates 
the array to contain Os at all locations which can be attacked by rooks, as shown in 
Figure 25.1(c). 

Hint: Make use of the first row and the first column. 

Solution: This problem is trivial with an additional n -bit array R and an additional 
ra-bit array C. We simply initialize R and C to Is. Then we iterate through all entries 
of A, and for each (i,j) such that A[i][j] = 0, we set R[i] and C[j] to 0. Consequently, 
a 0 in R[i] indicates that Row i should be set to 0; columns are analogous. A second 
iteration through all entries can be used to set the Os in A. 

The drawback with the above approach is the use of 0(n + m) additional storage. 
The solution is to use storage in A itself. The reason we can do this is because if a 
single 0 appears in a row, the entire row is cleared. Consequently, we can store a 
single extra bit r denoting whether Row 0 has a 0 within it. Now Row 0 can play the 
role of C in the algorithm in the previous paragraph. If we record a 0 in R[i], that is 
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the value we would be writing at that location, so the original entry is not lost. If R[i] 
holds a 1 after the first pass, it retains that value, unless r indicates Row 0 is to be 
cleared. Columns are handled in exactly the same way. 

public static void rookAttack(List<List<Integer>> A) { 
int m = A.size(), n = A . get (®) . size () ; 
boolean hasFirstRowZero = false; 
for (int j = ®; j < n; ++j) { 
if (A.get(®).get(j) == 0) { 
hasFirstRowZero = true; 
break; 

} 

} 

boolean hasFirstColumnZero = false; 
for (int i = ®; i < m; ++i) { 
if (A . get (i) . get (®) == ®) { 
hasFirstColumnZero = true; 
break; 

} 

} 

for (int i = 1; i < m; ++i) { 
for (int j = 1; j < n; ++j) { 
if (A.get(i).get(j) == ®) { 

A . get (i) . set (® , ®) ; 

A . get (®) . set ( j , ®) ; 

1 

} 

} 

for (int i = 1; i < m; ++i) { 
if (A. get (i) . get (®) == ®) { 

Collections.fill(A.get(i), ®); 

} 

} 

for (int j = 1; j < n; ++j) { 
if (A.get(®) . get (j) == ®) { 
for (int i = 1; i < m; ++i) { 

A.get(i).set (j , ®) ; 

} 

} 

} 

if (hasFirstRowZero) { 

Collections.fill(A.get(®) , ®) ; 

} 

if (hasFirstColumnZero) { 

for (int i = ®; i < m; ++i) { 

A . get (i) . set (® , ®) ; 

} 

} 


The time complexity is 0(mn) since we perform 0(1) computation per entry. The 
space complexity is 0(1). 
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25.8 Justify text © 

This problem is concerned with justifying text. It abstracts a problem arising in 
typesetting. The input is specified as a sequence of words, and the target line length. 
After justification, each individual line must begin with a word, and each subsequent 
word must be separated from prior words with at least one blank. If a line contains 
more than one word, it should not end in a blank. The sequences of blanks within 
each line should be as close to equal in length as possible, with the longer blank 
sequences, if any, appearing at the initial part of the line. As an exception, the very 
last line should use single blanks as separators, with additional blanks appearing at 
its end. 

For example, if A = ("The", "quick", "brown", "fox", "jumped", "over", 
"the", "lazy", "dogs.") and the line length L is 11, then the returned re¬ 
sult should be “The ^^quick”, “brown^^fox”, “ jumpecLover” , “the^^lazy”, 
“dogs.. The symbol ^ denotes a blank. 

Write a program which takes as input an array A of strings and a positive integer L, 
and computes the justification of the text specified by A. 

Hint: Solve it on a line-by-line basis, assuming a single space between pairs of words. Then 
figure out how to distribute excess blanks. 

Solution: The challenge in solving this problem is that it requires lookahead. Specifi¬ 
cally, the number of spaces between words in a line cannot be computed till complete 
set of words in that line is known. 

We solve the problem on a line-by-line basis. First, we compute the words that go 
into each line, assuming a single space between words. After we know the words in 
a line, we compute the number of blanks in that line and distribute the blanks evenly. 
The final line is special-cased. 


public static List<String> justifyText(String[] words, int L) { 
int currLineStart = ®, numWordsCurrLine = ®, currLineLength = ®; 
List<String> result = new ArrayList<>(); 
for (int i = ®; i < words.length; ++i) { 

// currLineStart is the first word in the current line , and i is used to 
// identify the last word. 

++numWordsCurrLine; 
int lookaheadLineLength 

= currLineLength + words[i].lengthQ + (numWordsCurrLine - 1); 
if (lookaheadLineLength == L) { 
result.add( 

joinALineWithSpace(words, currLineStart, i, i - currLineStart)); 
currLineStart = i + 1; 
numWordsCurrLine = ®; 
currLineLength = ®; 

} else if (lookaheadLineLength > L) { 

result.add(joinALineWithSpace(words, currLineStart, i - 1, 

L - currLineLength)); 

currLineStart = i; 
numWordsCurrLine = 1; 
currLineLength = words[i].length(); 

} else { // lookaheadLineLength < L. 
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currLineLength += words[i].length(); 

} 

} 

// Handles the last line. Last line is to be left - aligned. 
if (numWordsCurrLine > ®) { 

StringBuilder line = new StringBuilder(joinALineWithSpace( 

words, currLineStart, words.length - 1, numWordsCurrLine - 1)); 
for (int i = ®; i < L - currLineLength - (numWordsCurrLine - 1); i++) { 
line.append( ’ ’ ) ; 

} 

result.add(line.toString ()) ; 

} 

return result; 


// Joins strings in words[start : end] with numSpaces spaces spread evenly. 
private static String joinALineWithSpace(String[] words, int start, int end, 

int numSpaces) { 

int numWordsCurrLine = end - start + 1; 

StringBuilder line = new StringBuilder(); 
for (int i = start; i < end; ++i) { 
line.append(words[i]); 

--numWordsCurrLine; 

int numCurrSpace = (int)Math.ceil ((double) numSpaces / numWordsCurrLine); 
for (int j = ®; j < numCurrSpace; j++) { 
line.append( ’ ’ ) ; 

} 

numSpaces -= numCurrSpace; 

} 

line.append(words[end]) ; 
for (int i = ®; i < numSpaces; i++) { 
line.append(’ ’) ; 

} 

return line.toString(); 


Let n be the sum of the lengths of the strings in A. We spend 0(1) time per character 
in the first pass as well as the second pass, yielding an 0(n) time complexity. 

25.9 Implement list zipping © 

Let L be a singly linked list. Assume its nodes are numbered starting at 0. Define the 
zip of L to be the list consisting of the interleaving of the nodes numbered 0,1,2,... 
with the nodes numbered n - 1, n - 2, n - 3,..., where n is the number of nodes in the 
list. The zipping function is illustrated in Figure 4.1 on Page 27. 

Implement the zip function. 

Hint: Consider traversing the list in reverse order. 

Solution: A brute-force approach is to iteratively identify the head and tail, remove 
them from the original list, and append the pair to the result. The time complexity is 
0(n) + 0(n - 2) + 0(n - 4) + • • • = 0(n 2 ), where n is the number of nodes. The space 
complexity is 0(1). 
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The 0(n 2 ) complexity comes from having to repeatedly traverse the list to identify 
the tail. Note that getting the head of a singly linked list is an 0(1) time operation. 
This suggests paying a one-time cost of 0(n) to reverse the second half of the original 
list. Now all we need to do is interleave this with the first half of the original list. 

public static ListNode<Integer> zippingLinkedList(ListNodednteger> L) { 

if (L == null || L.next == null) { 
return L; 

} 

// Find the second half of L. 

ListNodecInteger> slow = L, fast = L; 

while (fast != null && fast.next != null) { 
fast = fast . next. next; 
slow = slow.next; 

} 

ListNodecInteger> firstHalfHead = L, secondHalfHead = slow.next; 

slow.next = null; // Splits the list into two lists. 

secondHalfHead = reverseLinkedList(secondHalfHead); 

// Interleave the first half and the reversed of the second half. 

ListNodecInteger> firstHalflter = firstHalfHead; 

ListNodecInteger> secondHalflter = secondHalfHead; 

while (secondHalflter != null) { 

ListNodecInteger> temp = secondHalflter.next; 
secondHalflter.next = firstHalflter.next; 
firstHalflter.next = secondHalflter; 
firstHalflter = firstHalflter.next.next; 
secondHalflter = temp; 

> 

return L; 


The time complexity is 0(n). The space complexity is 0(1). 


25.10 Copy a postings list Q< 

Postings lists are described in Problem 9.5 on Page 139. Implement a function which 
takes a postings list and returns a copy of it. You can modify the original list, but 
must restore it to its initial state before returning. 

Hint: Copy the jump field and then copy the next field. 

Solution: Here is a brute-force algorithm. First, create a copy of the postings list, 
without assigning values to the jump field. Next, use a hash table to store the mapping 
from nodes in the original postings list to nodes in the copied list. Finally, traverse 
the original list and the new list in tandem, using the mapping to assign each jump 
field. The time and space complexity are 0(n), where n is the number of nodes in the 
original postings list. 

The key to improving space complexity is to use the next field for each node in 
the original list to record the mapping from the original node to its copy. To avoid 
losing the structure of the original list, we use the next field in each copied node to 
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point to the successor of its original node. See Figure 25.2(b) for an example. Now we 
can proceed like we did in the brute-force algorithm. We assign the jump field in the 
copied nodes, using the next field in the original list to get the corresponding nodes 
in the copy. See Figure 25.2(c). Finally, we undo the changes made to the original list 
and update the next fields of the copied list, as in Figure 25.2(d) for an example. 






(a) Initial list. 


rn —►EH 


zmxi 



(b) Copy with cross-referencing. 







0— 


(d) Undo changes. 

Figure 25.2: Duplicating a postings list. 


public static PostingListNode copyPostingsList(PostingListNode L) { 
if (L == null) { 
return null; 

} 

// Stage 1: Makes a copy of the original list without assigning the jump 
// field, and creates the mapping for each node in the original 

// list to the copied list. 

PostingListNode iter = L; 
while (iter != null) { 

PostingListNode newNode 

= new PostingListNode(iter.order, iter.next, null); 
iter.next = newNode; 
iter = newNode.next; 

} 

// Stage 2: Assigns the jump field in the copied list. 
iter = L; 

while (iter != null) { 
if (iter.jump != null) { 

iter.next.jump = iter.jump.next; 

} 

iter = iter.next.next; 

} 

// Stage 3: Reverts the original list, and assigns the next field of 
// the copied list. 

iter = L; 

PostingListNode newListHead = iter.next; 
while (iter.next != null) { 
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PostingListNode temp = iter.next; 
iter.next = temp.next; 
iter = temp; 

} 

return newListHead; 


The time complexity is 0(n). The space complexity is 0(1). 


25.11 Compute the longest substring with matching parens 

Problem 9.3 on Page 137 defines matched strings of parens, brackets, and braces. This 
problem is restricted to strings of parens. Specifically, this problem is concerned with 
a long substrings of matched parens. As an example, if s is "((OXXOC*/ then "(OX)” is 
a longest substring of matched parens. 

Write a program that takes as input a string made up of the characters '(' and ')', and 
returns the size of a maximum length substring in which the parens are matched. 

Hint: Start with a brute-force algorithm and then refine it by considering cases in which you can 
advance more quickly. 

Solution: One approach would be to run the algorithm in Solution 9.3 on Page 137 on 
all substrings. The time complexity is 0(n 3 ), where n is the length of the string—there 
are (”) = substrings, and the matching algorithm runs in time 0(n). 

Note that if a prefix of a string fails the matched test because of an unmatched 
right parens, no extension of that prefix can be matched. Therefore, a faster approach 
is for each i to find the longest substring starting at the ith character that is matched. 
This leads to an 0(n 2 ) algorithm. 

Finally, if a substring ends in an unmatched right parens, but all of that substring's 
prefixes ending in right parens are matched, then no nonempty suffix of the prefix 
can be the prefix of a matched string, since any such suffix has fewer left parens. 
Therefore, as soon as a prefix has an unmatched right parens, we can continue with 
the next character after that prefix. We store the left parentheses' indices in a stack. 
At the same time, when we process a right parens for the given prefix, if it matched, 
we use the index at the top of the stack to update the longest matched string seen for 
this prefix. 

For the given example, "((OXXOC”/ we push left parentheses and pop on right 
parentheses. Before the first pop, the stack is (0,1,2), where the first array element 
is the bottom of the stack. The corresponding matched substring length is 3 - 1 = 2. 
Before the second pop, the stack is (0,1). The corresponding matched substring 
length is 4 - 0 = 4. Before the third pop, the stack is (0,5). The corresponding 
matched substring length is 6 - 0 = 6. Before the last pop, the stack is (0,7,8). The 
corresponding matched substring length is 9 - 7 = 2. 

public static int longestValidParentheses(String s) { 
int maxLength = Q, end = -1; 

Dequednteger> leftParenthesesIndices = new LinkedList <>() ; 
for (int i = ®; i < s.length(); ++i) { 
if (s.charAt(i) == ’(’) { 
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leftParenthesesIndices.addFirst(i); 

} else if (leftParenthesesIndices.isEmpty()) { 
end = i; 

} else { 

leftParenthesesIndices.removeFirst(); 

int start = leftParenthesesIndices.isEmpty() 

? end 

: leftParenthesesIndices .peekFirstO ; 
maxLength = Math.max(maxLength, i - start); 

} 

} 

return maxLength; 


The time and space complexity are 0(n). 


25.12 Compute the maximum of a sliding window © 

Network traffic control sometimes requires studying traffic volume over time. This 
problem explores the development of an efficient algorithm for computing the traffic 
volumes. 


3.7 



(a) Traffic at various timestamps. 

3.7 3.7 3.7 



(b) Maximum traffic over a window size of 3. 

Figure 25.3: Traffic profile before and after windowing. 


You are given traffic at various timestamps and a window length. Compute for each 
input timestamp, the maximum traffic over the window length time interval which 
ends at that timestamp. See Figure 25.3 for an example. 

Hint: You need to be able to identify the maximum element in a queue quickly. 
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Solution: Assume the input is specified by window length w and an array A of pairs 
consisting of integer timestamp and corresponding traffic volume. If A is not sorted 
by timestamp, we sort it. For example, the traffic in 25.3(a) on the preceding page cor¬ 
responds to the array <(0,1.3), (2,2.5), (3,3.7), (5,1.4), (6,2.6), (8,2.2), (9,1.7), (14,1.1)>. 

The brute-force algorithm entails finding for each input timestamp, the maximum 
in the subarray consisting of elements whose timestamps lie in the window ending at 
that timestamp. The time complexity is 0{mv), where n is the length of A. The reason 
is that every window may have up to w + 1 elements. 

The intuition for improving the time complexity of the brute-force approach stems 
from noting that as we advance the window only the boundary changes. Specifically, 
some older elements fall out of the window, and a new element is added. Therefore 
a queue is a perfect representation for the window. We need the maximum traffic 
within each window, which suggests using Solution 9.10 on Page 147. queue with 
maximum 

Initialize O to an empty queue with maximum. Iteratively enqueue (t u £,) in 
order of increasing i. For each i, iteratively dequeue Q until the difference of 
the timestamp at O's head and £, is less than or equal to w. The sequence of 
maximum values in the queue for each i is the desired result. For the input 
in Figure 25.3(a) on the preceding page with window size of 3, the output is 
((0,1.3), (2,2.5), (3,3.7), (5,3.7), (6,3.7), (8,2.6), (9,2.6), (14,1.7)). See Figure 25.3(b) on 
the facing page for a graphical representation. 

public static class TrafficElement implements Comparable<TrafficElement> { 

public int time; 

public double volume; 

public TrafficElement (int time, double volume) { 
this. time = time; 
this. volume = volume; 

} 

©Override 

public int compareTo(TrafficElement o) { 

int volumeCmp = Double.compare(volume, o.volume); 
return volumeCmp != © ? volumeCmp : time - o.time; 

} 

©Override 

public boolean equals(0bject o) { 
if (this == o) { 
return true; 

} 

if (o == null || getClass() != o. getClass()) { 

return false; 

} 

return compareTo((TrafficElement)o) == ®; 

} 

©Override 

public int hashCode() { return Objects.hash(volume, time); } 

} 
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public static ListcTrafficElement> computeTrafficVolumes( 

ListcTrafficElement> A, int w) { 

QueueWithMaxAlternative.QueueWithMaxcTrafficElement> slidingWindow 
= new QueueWithMaxAlternative.QueueWithMax<>(); 

ListcTrafficElement> maximumVolumes = new ArrayListc>(); 
for CTrafficElement trafficlnfo : A) { 
siidingWindow.enqueue(trafficlnfo); 

while (trafficlnfo.time - slidingWindow.head().time > w) { 
slidingWindow.dequeue(); 

} 

maximumVolumes.add( 

new TrafficElement(trafficlnfo.time, slidingWindow.max().volume)); 

} 

return maximumVolumes; 


Each element is enqueued once. Each element is dequeued at most once. Since 
the queue with maximum data structure has an 0(1) amortized time complexity per 
operation, the overall time complexity is 0(n). The additional space complexity is 
0(w). 


25.13 Implement a postorder traversal without recursion & 

This problem is concerned with traversing nodes in a binary tree in postorder fashion. 
See Page 151 for details and examples of these traversals. Generally speaking, a 
traversal computation is easy to implement if recursion is allowed. 

Write a program which takes as input a binary tree and performs a postorder traversal 
of the tree. Do not use recursion. Nodes do not contain parent references. 

Hint: Study the function call stack for the recursive versions. 

Solution: The brute-force approach to remove recursion from a function is to mimic 
the function call stack with an explicit stack. One of the challenges with this approach 
is to determine where to return to. 

Now we address the problem of implementing a postorder traversal without 
recursion. First we discuss a roundabout way of doing this. An inverted preorder 
traversal is the following: visit root, inverted preorder traverse the right subtree, 
then inverted preorder traverse the left subtree. For example, the inverted preorder 
traversal of the tree in Figure 10.1 on Page 150 visits nodes in the following order: 

(a,i,o,p,j,k,n,l,m,b,f,g,h,c,e,d). 

Intuitively, since the inverted preorder traversal is visit root, traverse right, traverse 
left, its reverse is traverse left, traverse right, visit root, i.e., the postorder traversal. 
Therefore, one way to compute the postorder traversal visit sequence without using 
recursion is to perform an inverted preorder traversal and instead of outputting nodes, 
we store them. When the inverted preorder traversal is complete, we iterate through 
the nodes in last-in, first-out order, which gives the postorder traversal sequence. 

The inverted preorder traversal itself can be performed nonrecursively using the 
solution to the first part of this problem, with the order in which the left and right 
children are pushed swapped. This algorithm is implemented in the code below. 


454 



public static List<Integer> postorderTraversal(BinaryTreeNode<Integer> tree) { 
List<Integer> sequence = invertedPreorderTraversal(tree); 

Collections.reverse(sequence); 
return sequence; 

} 

private static List<Integer> invertedPreorderTraversal( 

BinaryTreeNode<Integer> tree) { 

Deque<BinaryTreeNode<Integer>> path = new LinkedList<>(); 
path.addFirst(tree); 

Listdnteger> result = new ArrayList<>() ; 
while (!path.isEmpty()) { 

BinaryTreeNode<Integer> curr = path.removeFirst(); 
if (curr == null) { 
continue; 

} 

result.add(curr.data); 
path.addFirst(curr.left); 
path.addFirst(curr.right); 

} 

return result; 


The time and space complexity are both 0(n ), where n is the number of nodes in the 
tree. 

In addition to its being unintuitive, a more technical limitation of the approach 
given above is that it requires 0(n) additional space. If the result is to be returned as 
an array, this is unavoidable. However, if we are simply required to print the nodes, 
it is possible to reduce the space complexity to 0(h) where h is the height of the tree. 

We know that a recursive implementation of a postorder traversal takes 0(n) and 
0(h) space, so we should be able to achieve this complexity by using a stack to mimic 
the function call stack. One challenge is keeping track of where to return to, since 
there are two recursive calls. The function call stack keeps a return address, but 
instruction addresses are not accessible from user code. We can determine where to 
continue from by inspecting where the last visited node is relative to the current node. 
This is explained in more detail below. 

We maintain a stack of nodes which evolves exactly as the sequence of nodes that 
the recursive algorithm makes calls from. 

To determine when a nonleaf at the top of the stack is ready for visiting, we need 
to know if we are moving back up the tree, and if so, which side of this nonleaf we 
are returning from. If we are coming back up from the left, we do not want to push 
the left child again, but do want to push the right child, since we still need to mimic 
the second recursive call. If we are coming up from the right child both children have 
been visited, so we want to pop the stack and visit the nonleaf node. 

// We use stack and previous node pointer to simulate postorder traversal. 
public static List<Integer> postorderTraversal(BinaryTreeNode<Integer> tree) { 

if (tree == null) { // Empty tree. 
return Collections.emptyList(); 

} 
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Deque<BinaryTreeNodednteger>> path = new LinkedList<>(); 
BinaryTreeNode<Integer> prev = null; 
path.addFirst(tree); 

List<Integer> postorderSequence = new ArrayList<>(); 
while (!path.isEmpty()) { 

BinaryTreeNode<Integer> curr = path.peekFirst(); 
if (prev == null || prev.left == curr || prev.right == curr) { 
// Fife came down to curr from prev. 
if (curr.left != null) { // Traverse left. 
path.addFirst(curr.left); 

} else if (curr.right != null) { // Traverse right. 

path.addFirst(curr.right); 

} else { // Leaf node, so visit current node. 
postorderSequence.add(curr.data); 
path.removeFirst(); 

} 

} else if (curr.left == prev) { 

// Done with left, so now traverse right. 
if (curr.right != null) { // Visit right. 

path.addFirst(curr.right); 

} else { // // No right child, so visit curr. 
postorderSequence.add(curr.data); 
path.removeFirst() ; 

} 

} else { 

// Finished traversing left and right, so visit curr. 
postorderSequence.add(curr.data); 
path.removeFirst(); 

} 

prev = curr; 

> 

return postorderSequence; 

} 


The time complexity is 0(n), since we perform a constant amount of work per node 
(a push and a pop). The space complexity is 0(h), since the stack corresponds to a 
path starting at the root. 


25.14 Compute fair bonuses ©• 

You manage a team of developers. You have to give concert tickets as a bonus to 
the developers. For each developer, you know how many lines of code he wrote the 
previous week, and you want to reward more productive developers. 

The developers sit in a row. Each developer, save for the first and last, has two 
neighbors. You must give each developer one or more tickets in such a way that if 
a developer has written more lines of code than a neighbor, then he receives more 
tickets than his neighbor. 

Your task is to develop an algorithm that computes the minimum number of tickets 
you need to buy to satisfy the constraint. For example, if Andy, Bob, Charlie, and 
David sit in a row from left to right, and they wrote 300, 400, 500, and 200 lines of 
code, respectively, the previous week, then Andy and David should receive one ticket 
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each. Bob should receive two tickets, and Charlie should receive three tickets, for a 
total of seven tickets. 

Write a program for computing the minimum number of tickets to distribute to the 
developers, while ensuring that if a developer has written more lines of code than a 
neighbor, then he receives more tickets than his neighbor. 

Hint: Consider iteratively improving an assignment that may not satisfy the constraint. 

Solution: A brute-force approach is to start by giving each developer a ticket. Next 
we perform the following iteration. We check if all developers are satisfied. If they 
are all satisfied, we are done. Otherwise, if some developer is not satisfied, i.e., he 
has written more lines of code than a neighbor, but does not have more tickets than 
that neighbor, we give him one more ticket than his neighbor. This approach uses 
the minimum number of tickets initially, and every additional ticket that is given is 
necessary. The time complexity is 0(kn 2 ), where k is the maximum number of tickets 
given to any single developer, and n is the number of developers. 

The key insight to a better algorithm is the observation that the least productive 
developer never needs to be given more than a single ticket. We can propagate this 
observation by processing developers in increasing order of productivity. Subse¬ 
quently, when we process a developer if his neighbors have been processed, he must 
be at least as productive as them. If a developer is more productive than a neighbor, 
he must be given at a minimum one more ticket than that neighbor. If a developer 
is only as productive as his neighbors, we only need give him the same number of 
tickets, as per the problem specification. 

For the given example, our algorithm starts by giving 1 ticket to David. Andy is 
next in order of productivity, so we give him 1 ticket, since he has only one neighbor, 
who is more productive than him. Next we process Bob. He is more productive than 
Andy so we give him 1 + 1 = 2 tickets. Then comes Charlie. He is a neighbor of 
Charlie and David, and is more productive than both, so we give him max(2,1) +1 =3 
tickets, for a total of 7 tickets. 

This approach yields the correct result because once a developer is processed, we 
only process developers who are at least as productive in the future, meaning that 
once his bonus is assigned, it will never need updating in the future. Furthermore, 
any bonus that we assign is forced upon us by the problem constraints. 

A min-heap is a suitable data structure for processing the developers, and is used 
in the following program. 

private static class EmployeeData { 
public Integer productivity; 
public Integer index; 

public EmployeeData(Integer productivity, Integer index) { 
this .productivity = productivity; 
this. index = index; 

} 

} 

private static final int DEFAULT_INITIAL_CAPACITY = 16; 
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public static List<Integer> calculateBonus(List<Integer> productivity) { 
PriorityQueue<EmployeeData> minHeap = new PriorityQueue<>( 

DEFAULT_INITIAL_CAPACITY, new Comparator<EmployeeData>() { 

©Override 

public int compare(EmployeeData ol, EmployeeData o2) { 

int result = Integer.compare(ol.productivity, o2.productivity); 
if (result == Q) { 

result = Integer.compare(ol.index, o2.index); 

} 

return result; 

} 

}); 

for (int i = 0; i < productivity.size(); ++i) { 

minHeap.add(new EmployeeData(productivity.get(i), i)); 

} 

// Initially assigns one ticket to everyone. 

Listdnteger> tickets 

= new ArrayList<>(Collections.nCopies(productivity.size(), 1)); 

// Fills tickets from lowest rating to highest rating. 
while (!minHeap.isEmpty()) { 

EmployeeData p = minHeap.peek(); 
int nextDev = minHeap.peek().index; 

// Handles the left neighbor. 
if (nextDev > ®) { 

if (productivity.get(nextDev) > productivity.get(nextDev - 1)) { 
tickets.set(nextDev, tickets.get(nextDev - 1) + 1) ; 

} 

} 

// Handles the right neighbor. 
if (nextDev + 1 < tickets.size()) { 

if (productivity.get(nextDev) > productivity.get(nextDev + 1)) { 
tickets.set(nextDev, Math.max(tickets.get(nextDev), 

tickets.get(nextDev + 1) + 1)); 

} 

} 

minHeap.remove(p); 

} 

return tickets; 

} 

Since each extraction from a min-heap takes time 0(logn), the time complexity is 
0(n 1 ogn). 

The approach presented above is in the spirit of a brute-force solution. On some 
reflection, a total ordering on the developers is overkill, since the specified constraint 
is very local. Indeed, we can improve the time complexity to 0(n) by making two 
passes over the array. 

We start by giving each developer a single ticket. Then we make a left-to-right 
pass in which we give each developer who has more productivity than the developer 
on his left one ticket more than the developer on his left. We then do the same in a 
right-to-left pass. 

Any amount added is required, so we cannot get by with fewer tickets. Note that 
every developer who is more productive than his right neighbor has more tickets than 
that neighbor. Furthermore, if a developer is more productive than his left neighbor. 
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in the left-to-right pass we already give him more tickets than his left neighbor, and 
we can only increase his ticket count in the right-to-left pass. 

public static List<Integer> calculateBonus(List<Integer> productivity) { 

// Initially assigns one ticket to everyone. 

Listdnteger> tickets 

= new ArrayList<>(Collections.nCopies(productivity.size(), 1)); 

// From left to right. 

for (int i = 1; i < productivity.size(); ++i) { 

if (productivity.get(i) > productivity.get(i - 1)) { 
tickets.set(i, tickets.get(i - 1) + 1); 

} 

} 

// From right to left. 

for (int i = productivity.size() - 2; i >= ®; --i) { 
if (productivity.get(i) > productivity.get(i + 1)) { 

tickets.set(i, Math.max(tickets.get(i), tickets.get(i + 1) + 1)); 

} 

} 

return tickets; 


25.15 Search a sorted array of unknown length & 

Binary search is usually applied to an array of known length. Sometimes, the array is 
"virtual", i.e., it is an abstraction of data that is spread across multiple machines. In 
such cases, the length is not known in advance; accessing elements beyond the end 
results in an exception. 

Design an algorithm that takes a sorted array whose length is not known, and a key, 
and returns an index of an array element which is equal to the key. Assume that an 
out-of-bounds access throws an exception. 

Hint: Can divide and conquer be used to find the end of the array? 

Solution: The brute-force approach is to iterate through the array, one element at a 
time, stopping when either the key is found or an exception is thrown (in which case 
the key is not present). The time complexity is 0(n), where n is the length of the input 
array. 

A better approach is to take advantage of sortedness. If we know the array length, 
we can use binary search to search for the key. We can compute the array length by 
testing whether indices 0,1,3,7,15,... are valid. As soon as we find an invalid index, 
say 2' -1, we can use binary search over the interval [2 ,_1 , 2 l - 2] to find the first invalid 
index, which is the length of the array. 

We can improve on the above approach by comparing the value of the element at 
index 2' -1 with the key, since if it is greater than the key, we can do binary search over 
indices [2 ?_1 ,2' - 2] for the key. Conceptually, we can treat out-of-bounds indices in 
the same way as valid indices by treating an out-of-bounds index as holding infinity. 

For example, consider the array in Figure 12.1 on Page 190. Suppose we are 
searching for the key 243. We examine indices 0, 1, 3, 7. Since A[7] = 285 > 243, we 


459 



now perform conventional binary search over the interval [4,6] for 243. If instead, 
we were searching for the key 400, we would examine indices 0,1,3, 7,15. Since 15 is 
not a valid index, we would stop, and perform conventional binary search over the 
interval [8,14]. The first midpoint is 11, which is out-of-bounds and treated as holding 
infinity, so we update the interval to [8,10]. The next interval is [8,8], followed by 
[8,7] which is empty, indicating that the key 400 is not present. 

public static int binarySearchUnknownLength(List<Integer> A, int k) { 

// Find a range where k exists, if it’s present. 
int p = Q; 
while (true) { 
try { 

int idx = (1 « p) - 1; // 2 A p - 1. 
if (A.get(idx) == k) { 
return idx; 

} else if (A.get(idx) > k) { 
break; 

} 

} catch (IndexOutOfBoundsException e) { 
break; 

} 

++p; 

} 

// Binary search between indices 2 A (p - 1) and 2 A p - 2, inclusive. 
int left = Math.maxC®, 1 « (p - 1)), right = (1 « P) - 2; 
while (left <= right) { 

int mid = left + ((right - left) / 2); 
try { 

if (A.get(mid) == k) { 
return mid; 

} else if (A.get(mid) > k) { 
right = mid - 1; 

} else { // A.get(mid) < k 
left = mid + 1; 

} 

} catch (Exception e) { 

right = mid - 1; // Search the left part if out-of-bound. 

} 

} 

return -1; // Nothing matched k. 

} 


The run time of the first loop is <9(log n), since we double the tested index with each 
iteration. The second loop is conventional binary search, i.e., <9(log n), so the total 
time complexity is <9(log n). 


25.16 Search in two sorted arrays ©• 

You are given two sorted arrays and a positive integer k. Design an algorithm for 
computing the kth smallest element in an array consisting of the elements of the initial 
two arrays arranged in sorted order. Array elements may be duplicated within and 
across the input arrays. 
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Hint: The first k elements of the first array together with the first k elements of the second array 
are initial candidates. Iteratively eliminates a constant fraction of the candidates. 

Solution: You could merge the two arrays into a third sorted array and then look for 
the answer—the merge would take 0(m + n) time, where m and n are the lengths of 
the input arrays. 

You can optimize somewhat by building the merged array on the first k elements, 
which would be an 0(k) operation—this is faster than forming the combined array 
when k is small, but when k is comparable to m and n, the time complexity is 0(m + n). 

What we really need is some form of binary search that takes advantage of the 
sortedness of A and B. Intuitively, if we focus on finding the indices in A and B that 
correspond to the first k elements, we stand a good chance of using binary search. 
Specifically, suppose the first k elements of the union of A and B consist of the first 
x elements of A and the first k - x elements of B. We'll now see how to use binary 
search to determine x. 

Let's maintain an interval [ b , t] that contains x. The iteration continues as long 
as b < t. We will contract this interval by half in each iteration. At each iteration 
consider the midpoint, x = b + |_^J. If A[x] < B[(k — x) — 1], then A[x] must be in the 
first k - 1 elements of the union, so we update b to x + 1 and continue. Similarly, if 
A[x— 1] > B[k- x], then A[x- 1] cannot be in the first k elements, so we can update t to 
x-1. Otherwise, we must have B[(k - x) - 1] < A[x] and A[x - 1] < B[k - x], in which 
case the result is the larger of A[x - 1] and B[(k - x)-l], since the first x elements of 
A and the first k-x elements of B when sorted end in A[x - 1] or B[(k - x) - 1]. 

If the iteration ends without returning, it must be that b = t . Clearly, x = b = t. We 
simply return the larger of A[x - 1] and B[(k - x) - 1]. (If A[x - 1] = B[(k -x)- 1], we 
arbitrarily return either.) 

The initial values for b and t need to be chosen carefully. Naively setting b = 0, t = k 
does not work, since this choice may lead to array indices in the search lying outside 
the range of valid indices. The indexing constraints for A and B can be resolved by 
initializing b to max(0, k- n) and t to min(m, k). 

public static int findKthNTwoSortedArrays(Listdnteger> A, List<Integer> B, 

int k) { 

// Lower bound of elements we will choose in A. 

int b = Math.max(®, k - B.sizeO); 

// Upper bound of elements we will choose in A. 

int t = Math.min(A.size(), k); 

while (b < t) { 

int x = b + C(t - b) / 2) ; 

int ax 1 = (x <= ® ? Integer.MIN.VALUE : A.get(x - 1)); 
int ax = (x >= A.sizeO ? Integer. MAX_VALUE : A.get(x)); 
int bkxl = (k - x <= ® ? Integer.MIK.VALUE : B.get(k -x-1)); 
int bkx = (k - x >= B.sizeO ? Int eger. MAX_VALUE : B.get(k - x)) ; 

if (ax < bkxl) { 
b = x + 1; 

} else if (axl > bkx) { 
t = x - 1; 

} else { 


461 



// B.get(k - x - 1) <= A.get(x) && A.get(x - 1) < B.get(k - x). 
return Math.max(axl, bkxl); 

} 

} 

int abl = b <= ® ? Integer.MIN_VALUE : A.get(b - 1); 

int bkbl =k-b-l<®? Integer.MIN.VALUE : B.get(k - b - 1); 

return Math.max(abl, bkbl); 


Since in each iteration we halve the length of [b, t] the time complexity is <9(log k). 


25.17 Find the km largest element—large n, small k O’ 

The goal of this problem is to design an algorithm for computing the kth largest 
element in a sequence of elements that is presented one element at a time. The length 
of the sequence is not known in advance, and could be very large. 

Design an algorithm for computing the kth largest element in a sequence of elements. 

Hint: Track the k largest elements, but don't update the collection immediately after each new 
element is read. 

Solution: The natural approach is to use a min-heap containing the k largest elements 
seen thus far, just as in Solution 11.4 on Page 181. When the last element in the 
sequence is read, the desired value is the element at the root of the min-heap. This 
approach has time complexity 0(n log k), where n is the total number of elements in 
the sequence. 

We know of a very fast algorithm for finding the kth largest element in an array 
of fixed size (Solution 12.8 on Page 200). We cannot directly apply that here without 
allocating 0(n) space. However, we can break our input into fixed size arrays and run 
Solution 12.8 on Page 200 over those arrays to eliminate all but the k largest elements 
from each array. These elements are added over to the next array. 

public static int findKthLargestUnknownLength(Iterator<Integer> sequence, 

int k) { 

ArrayListdnteger> candidates = new ArrayList<>(2 * k - 1); 
while (sequence.hasNext()) { 
int x = sequence.next() ; 
candidates.add(x); 

if (candidates.size() == 2 * k - 1) { 

// Reorders elements about median with larger elements appearing before 
// the median. 

OrderStatistic.findKthLargest(candidates, k); 

// Resize to keep just the k largest elements seen so far. 
candidates.subList(k, candidates.size()).clear(); 

} 

} 

// Finds the k-th largest element in candidates. 

OrderStatistic.findKthLargest(candidates, k); 
return candidates.get(k - 1); 
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By using 2k - 1 as the array size, the time complexity to find the A:th largest element is 
almost certain 0(k). It is run every k- 1 elements, implying an 0(n) time complexity. 

Note that we could use less storage, e.g., an array of length 3k/2, and still achieve 
0(n) time complexity The actual run time would be higher with an array of length 
3k /2 since we only discard k/2 elements for each call to finding the kth. largest element. 
This is a classic space-time trade-off. If we used a 4 k long array, we could discard 3k 
elements for one call to Solution 12.8 on Page 200. The time complexity of Solution 12.8 
on Page 200 is proportional to the length of the array, so using a length 4 k array 
compared to a length 3k/2 array yields a speed-up of (1/ ^]/ 3) = 2.25. Clearly more 
storage leads to faster run times (in the extreme we read all n and do a single call to 
Solution 12.8 on Page 200), so there is a trade-off with respect to how much storage 
we want. 


25.18 Find an element that appears only once 

Given an integer array of length n, where each element except for one appears twice, 
with the remaining element appearing only once, we can use 0(n) space and 0(n) 
time to find the element that appears exactly once, e.g., using a hash table. However, 
there is a better solution: compute the bitwise-XOR (©) of all the elements in the array. 
Because x © x = 0, all elements that appear an even number of times cancel out, and 
the element that appears exactly once remains. Therefore, this problem can be solved 
using 0(1) space. 

Given an integer array, in which each entry but one appears in triplicate, with the 
remaining element appearing once, find the element appearing once. For example, if 
the array is (2, 4,2,5,2,5,5), you should return 4. 

Hint: Count the number of Is at each index. 

Solution: The brute-force solutions in Solution 25.18, namely using a hash table or 
sorting will work for this problem too, with the same time and space complexities. 

One way to view Solution 12.10 on Page 204 is that it counts modulo 2 for each 
bit-position the number of entries in which the bit in that position is 1. Specifically, 
the XOR of elements at indices [0, i — 1], determines exactly which bit-positions have 
been odd number of times in elements of the input array whose indices are in [0, i- 1]. 

The analogous approach for the current problem is to count modulo 3 for each 
bit-position the number of times the bit in that position has been 1. The effect of 
counting modulo 3 is to eliminate the elements that appear three times, and so the 
bit-positions which have a count of 1 are precisely those bit-positions in the count 
which are set to 1. 

The example array, (2,4,2,5,2,5,5), expressed in binary is ((010) 2 , (100) 2 , (010) 2 , 
(101) 2 , (010) 2 , (101) 2 , (101) 2 ). The number of bits set to 1 in position 0 (the LSB) across 
all 7 array entries is 3; the number of bits set to 1 in position 1 is 3, and the number of 
bits set to 1 in position 2 is 4. By taking each of these quantities modulo 3 we cast out 
the contributions of elements that appear exactly three times, which leaves us with 
a 1 in the MSB and 0 in the remaining two positions, i.e., the element which only 
appeared once. 
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We can implement the above idea using an array C of integers whose length equals 
the integer word size. Entry C[i] will be used to count the number of Is in bit-position 
i, across all the inputs. By the above argument, after the entire input is processed, 
C[i] mod 3 will be 1 at exactly those bit-positions where the input that appears once 
has a 1. 

public static int findElementAppearsOnce(List<Integer> A) { 
int[] counts = new int [32]; 
for (int x : A) { 

for (int i = Q; i < 32; ++i) { 
if ((x & (1 « i)) != ®) { 

++counts[i]; 

} 

} 

} 

int result = ®; 

for (int i = ®; i < 32; ++i) { 

result |= (counts[i] % 3) * (1 « i) ; 

} 

return result; 


The time complexity is 0(n) and space complexity is 0( 1). 

Variant: Solve the same problem when one entry appears twice and the rest appear 
three times. 


25.19 Find the line through the most points Q* 

You are given a set of points in the plane. Each point has integer coordinates. Design 
an algorithm for computing a line that contains the maximum number of points in 
the set. 

Hint: A line can be uniquely represented by two numbers. 

Solution: This problem may seem daunting at first—there are literally infinitely many 
lines. The only lines we care about are those that pass through points in the set, and, 
more specifically, lines that pass through at least two points in the set. 

A brute-force approach then is to compute all such lines, and for each such line, 
count exactly how many points from the set lie on it. We can use a hash table to 
represent the set of lines. The set of points corresponding to a line could itself be 
stored using a hash table. If there are n points in the set, there are n(n - l)/2 pairs of 
points. Naively, we would compute the set of lines defined by pairs of points, and 
then iterate over all points, checking for each line if that point belongs to that line. 
The time complexity is dominated by the iteration over points and lines, which is 
0(n x n(n - l)/2) = 0(n 3 ). 

A better approach is to add the pair of points to the set of points on the line they 
define immediately, for each pair we have to do a lookup, an insert into the hash 
table if the defined line is not already present, and two inserts into the corresponding 
set of points. The hash table operations are 0(1) time, leading to an 0(n 2 ) time 
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complexity for this part of the computation. The space complexity is also 0(n 2 ). This 
is a consequence of the time complexity. At a first glance, it seems like the space 
complexity might be higher, since there are 0(n 2 ) pairs of lines, and each can have up 
to 0(n) points. However, there is an inverse relationship between the number of lines 
and the number of points per line. 

We finish by finding the line with the maximum number of points with a simple 
iteration through the hash table searching for the line with the most points in its 
corresponding set. There are at most n(n - 1)/2 lines, so the iteration takes 0(n 2 ) time, 
yielding an overall time bound of 0(n 2 ). 

The design of a hash function appropriate for lines is more challenging than it may 
seem at first. The equation of the line through (x\,y\) and ( x 2 , yi) is 

_ yi-yi x 2 y 1 - x^ 2 

y X 2 -X 1 X X 2 -X! 

One idea would be to compute a hash code from the slope and the Y-intercept 
of this line as an ordered pair of floating-point numbers. However, because of finite 
precision arithmetic, we may have three points that are collinear map to distinct 
buckets. 

A more robust hash function treats the slope and the Y-intercept as rationals. A 
rational is an ordered pair of integers: the numerator and the denominator. We need 
to bring the rational into a canonical form before applying the hash function. One 
canonical form is to make the denominator always nonnegative, and relatively prime 
to the numerator. Lines parallel to the Y-axis are a special case. For such lines, we 
use the X-intercept in place of the Y-intercept, and use J as the slope. 

private static class Line { 

private static class Rational { 
public Integer numerator; 
public Integer denominator; 

public Rational(Integer numerator, Integer denominator) { 
this .numerator = numerator; 
this.denominator = denominator; 

} 

} 

public static Rational getCanonicalFractional(int a, int b) { 

int gcd = Biglnteger.valueOf(a).gcd(BigInteger.valueOf(b)).intValue(); 
a /= gcd; 
b /= gcd; 

return b < ® ? new Rational (-a, -b) : new Rational (a, b) ; 

} 

// Slope is a rational number. Note that if the line is parallel to y-axis 
// that we store I/O. 
private Rational slope; 

// Intercept is a rational number for the y-intercept unless 

// the line is parallel to y-axis in which case it is the x-intercept 

private Rational intercept; 

Line(Point a, Point b) { 
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if (a.x != b.x) { 

slope = getCanonicalFractional(b.y - a.y, b.x - a.x); 

} else { 

slope = new Rational (1, ®) ; 

} 

if (a.x != b.x) { 

intercept = getCanonicalFractional(b.x * a.y - a.x * b.y, b.x - a.x) 
} else { 

intercept = new Rational(a. x , 1); 

} 

} 

©Override 

public boolean equals(Object obj) { 

if (obj == null || !(obj instanceof Line)) { 

return false; 

} 

if (this == obj) { 
return true; 

} 

Line that = (Line)obj ; 

return slope.equals(that.slope) && intercept.equals(that.intercept); 


} 


©Override 

public int hashCode() { 

return Objects.hash(slope.numerator, slope.denominator, 

intercept.numerator, intercept.denominator); 


} 


public static Line findLineWithMostPoints(List<Point> P) { 
// Adds all possible lines into hash table. 

MapcLine, Set<Point>> table = new HashMap<>(); 
for (int i = Q; i < P.size(); ++i) { 

for (int j = i + 1; j < P.size(); ++j) { 

Line 1 = new Line(P.get(i), P.get(j)); 


Set<Point> si = table . get(1) ; 
if (si == null) { 

si = new HashSet<>(); 
table.put(1, si) ; 

} 

si.add(P.get(i)) ; 


Set<Point> s2 = table . get (1) ; 
if (s2 == null) { 

s2 = new HashSet<>(); 
table.put(1, s2) ; 

} 

s2.add(P.get (j)) ; 

} 

} 


return Collections 

.max(table.entrySet(), 
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new Comparator<Map.EntrycLine, Set<Point>>>() { 
©Override 

public int compare(Map.EntrycLine, Set<Point>> el, 

Map . Entry<Line , Set<Point» e2) { 
if (el != null && e2 != null) { 

return Integer.compare(el.getValue().size(), 
e2.getValue().size()) ; 

} 

return (el != null ? 1 : -1); 

} 

}) 

.getKey () ; 

} 


25.20 Find the shortest unique prefix 

It is natural to speak of prefixes and suffixes of strings. For example, the prefixes 
of "banana" are "" (the empty string), "b", "ba", "ban", "bana", "banan", "banana". 
The suffixes of "banana" are "banana", "anana", "nana", "ana", "na", "a", "". 

This problem is concerned with finding the shortest prefix of a string (the query 
string) that is not in a set of strings (the dictionary strings). If all prefixes of the string 
are present, return the empty string. For example: 

• The query is "cat", the dictionary is {"dog", "be", "cut"}: return "ca" since the 
prefixes of "cat" are "c", "ca", and "cat", and "c" is a prefix of a word in the 
dictionary, but "ca" is not. 

• The query is "cat", the dictionary is {"dog", "be", "cut", "car"}: return "cat" 
"c" and "ca" are prefixes of words in the dictionary, but "cat" is not. 

Write a program that takes as input a query string and a dictionary, i.e., a nonempty 
set of strings, and returns the shortest prefix of the query string which is not a prefix 
of any string in the dictionary. Assume all strings are nonempty. Return the empty 
string if all prefixes of the query are prefixes of some string in the dictionary. For 
example, if the query string is "cat", the dictionary is {"dog", "be", "cut", "car", 
"catsnip", "category"}, your program should return "", since all prefixes of "cat" are 
prefixes of words in the dictionary. 

Hint: How would you represent a set of strings as a rooted tree? 

Solution: First, note that this problem only makes sense if the function is called many 
times with different query strings and an unchanging set of dictionary strings. Other¬ 
wise, there is nothing that can improve upon the brute-force approach of comparing 
the query string with each dictionary string. 

If the dictionary is fixed, we can improve upon the brute-force approach by pre¬ 
processing the dictionary strings into bins, which are subsets of the dictionary strings 
based on their initial character. This way, we do not need to compare the query string 
with all dictionary strings, just those that begin with the same character. If there are 
no dictionary words starting with the initial character of the query string, the length-1 
prefix of the query string is the desired string. Otherwise, we need to solve the same 
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problem with the suffix of the query string and the suffixes of the dictionary strings 
in the corresponding bucket. 

For example, if the query string is "cat" and the {"dog", "be", "cut", "car"), the 
bins are {"be"), {"car", "cut"}, and {"dog"}. We focus on the set {"car", "cur"}, and 
look for the longest prefix of "at" not in {"ar", "ur"}. 

Generalizing this idea, we can recursively bin the subsets, which leads us to the idea 
of the trie. This is a data structure for storing a set of strings based on positional trees. 
To be concrete, suppose the strings are over the alphabet {"a", "b", ..., "z"}. Each 
node has a hash table mapping each character in the alphabet to the corresponding 
child pointer. Some or all of the children may be null. A path of length l starting from 
the root naturally corresponds to a string of / characters. Each node has a Boolean 
field indicating whether the string corresponding to the path from the root is a string 
in the set. 

Finding a shortest prefix of s that is not a prefix of any string in the represented 
set is simply a matter of finding the first node m on the search path from the root that 
does not have a child corresponding to the next character in s. 

public static String findShortestPrefix(String s, Set<String> D) { 

// Builds a trie according to given dictionary D. 

Trie T = new Trie(); 
for (String word : D) { 

T.insert(word); 

} 

return T.getShortestUniquePrefix(s); 

} 

public static class Trie { 

private static class TrieNode { 
private boolean isString = false; 

private MapcCharacter, TrieNode> leaves = new HashMapoQ ; 
public boolean getlsString() { return isString; } 
public void setlsString (boolean string) { isString = string; } 
public MapcCharacter, TrieNode> getLeaves() { return leaves; } 

} 

private TrieNode root = new TrieNode(); 

public boolean insert(String s) { 

TrieNode p = root; 

for (int i = Q; i < s.length(); i++) { 
char c = s.charAt(i); 
if (!p.getLeaves().containsKey(c)) { 
p.getLeaves().put(c, new TrieNodeO); 

} 

p = p.getLeaves().get(c); 

} 

// s already existed in this trie. 
if (p.getlsString()) { 

return false; 


468 



} else { // p. getlsString () == false 

p.setlsString (true); // Inserts s into this trie. 

return true; 

} 

} 

public String getShortestUniquePrefix(String s) { 
TrieNode p = root; 

StringBuilder prefix = new StringBuilder(); 
for (int i = Q; i < s.length(); i++) { 
char c = s.charAt(i); 
prefix.append(c) ; 

if (!p.getLeaves().containsKey(c)) { 
return prefix.toString(); 

} 

p = p.getLeaves() . get (c) ; 

} 

return 

} 

} 


The time complexity to construct the trie is 0(\D\L) where L is the length of the longest 
string in D. The query time takes <9(min(L, n))), where n is the length of s. 

Variant: How would you find the shortest string that is not a prefix of any string in 
D? 

25.21 Find the most visited pages in a window 

This problem is a continuation of Problem 15.8 on Page 269. The difference is that 
each line includes a visit-time and only pages whose visit-times are no older than 
a specified duration (the "window size") of the visit-time of the most recently read 
page are to be considered. The window size is specified at the beginning, and never 
changes. 

Implement the API in Problem 15.8 on Page 269, with the following update: compute 
the k most visited pages from the pages whose visit-time is within W of the most 
recent page's visit-time. Here W, the window size, is an input parameter fixed for a 
given log file. You can assume visit-times increase as you process the file. 

Hint: Use a federation of data structures. 

Solution: A brute-force approach would be to store all (page,visit-time) pairs and 
restrict the query to the pairs whose visit time is no more than W less than the most 
recent visit-time. 

We can reduce RAM usage by noting that once a line in the log file has a visit-time 
that is too old, we do not need to return to that entry ever again. The natural way to 
"age out" (page,visit-time) pairs is to store them in a queue. Whenever we add a new 
(page,visit-time) pair, it goes to the head of the queue, and we pop entries which are 
more than W delayed with respect to the visit-time. 

The rest of the algorithm is very similar to Solution 15.8 on Page 270. We use a BST 
to store (page,visit-count) pairs, ordered by visit-counts, breaking ties on pages. We 
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keep a hash table mapping pages to their (page,visit-count) pair. Each time a page 
visit is evicted from the queue, we decrement that page's visit-count, updating the 
BST. 

The time complexity for adding pages is dominated by updates to the BST. In the 
worst-case, every page is unique and all appear in the window, leading to an 0(n log n) 
time complexity, where n is the number of log entries. The space complexity is 0{n). 

In practical settings, the maximum number of pages in a window, and hence in the 
BST, will likely be much less than n. If the number of entries in a window is no more 
than c, then the time complexity for n calls to the add function is 0(n log c) —the log c 
term corresponds to the time needed to perform BST updates. The space complexity 
is <9(c). 

As a concrete example, let the log file contain the entries 
(a, 11), (a, 11), (g, 15), (f,16),(t,17),(fl,18),(t,21). Let the 
window duration be 4. Then after the first five entries have been read, the BST 
contains the following keys, in this order: (l,a), (2, t). If k = 1, i.e., we want the most 
common page in the window, we simply return t. The queue contains (t, 3), (£, 6), {a, 7), 
from tail to head. After the next entry, (a, 11) is read, the elements (f, 3) and (f, 6) are 
popped (since the visit-times 3 and 6 are earlier than 11-4 = 7. Entry {a, 11) is added 
to the queue. The BST now contains just (2, a). 

Variant: Solve the same problem when the logfile can contain multiple visits to a page 
at the same time, and entries can be out-of-order with respect to visit-time. 

Variant: Write a program for the same problem with <9(1) time complexity for the 
read function and 0(k) time complexity for the find function. 

25.22 Convert a sorted doubly linked list into a BST & 

Lists and BSTs are both examples of "linked" data structures, i.e., some fields are 
references to other objects of the same type. Since nodes in a doubly linked list and in 
a BST both have a key field and two references, it's natural to consider the following 
problem. 

Write a program that takes as input a doubly linked list of sorted numbers and builds 
a height-balanced BST on the entries in the list. Reuse the nodes of the list for the 
BST, using the previous and next fields for the left and right children, respectively. 
See Figure 25.4(b) on the facing page for an example of a doubly linked list, and 
Figure 25.4(a) on the next page for the BST on the same nodes. 

Hint: Update reference fields, not node contents. 

Solution: If the list nodes were in an array, we could index directly into the array 
to obtain the midpoint, and the time complexity would satisfy T(n ) = (9(1) + 2T(|), 
where n is the number of nodes in the list, which solves to T(n) = 0(n). This is the 
approach of Solution 15.9 on Page 271. We can recycle the list nodes, but creating the 
array entails 0(n) additional space. 

A direct approach to the construction which does not allocate new nodes is to find 
the midpoint of the list, and use it as the root, recursing on the first half and the second 
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(a) A BST on five nodes—edges that do not terminate in nodes denote 
empty subtrees. The number in hex adjacent to each node represents 
its address in memory. 


H- Hx|2 I I 3 I 4 * > |*|5|.| « > 1*1 7 |. I - » [~* 111 |x| 

8x285® 8x184® 8x246® 8x1888 8x143® 

(b) The sorted doubly linked list corresponding to the BST in (a). Note how the tree nodes have been used for the 
list nodes. 


Figure 25.4: BST and sorted doubly linked list interconversion. 


half of the list. The time complexity satisfies the recurrence T(n ) = 0(n) + 2T(|)— 
the 0(n) term comes from the traversal required to find the midpoint of the list, 
which itself entails computing the length of the list. This solves to T(n) = 0(n log n). 
The added time complexity compared to the array-based approach comes from the 
inability to find a midpoint in a list in <9(1) time. 

The key insight to improving the time complexity without adding to the space 
complexity is noting that since we have to spend 0(n) time to find the midpoint, we 
should do more than just get the midpoint. Specifically, we can first create a balanced 
BST on the first [_§ J nodes. Then we use the (Lf J + l)th node as the root of the final 
BST and set its left child to the BST just created. 

Since we are changing the links in the list, we need to be careful to ensure we can 
recover the root. We can do this by keeping a reference to the head of the list being 
processed, and advancing this reference inside the recursive calls. This allows us to 
compute the root while computing the left subtree. 

Finally we create a balanced BST on the remaining n — |_ \ J - 1 nodes, and set it as 
the root's right child. 

private static DoublyLinkedListNode<Integer> head; 

// Returns the root of the corresponding BST. The prev and next fields of the 
// list nodes are used as the BST nodes left and right fields, respectively. 

// The length of the list is given. 

public static DoublyLinkedListNode<Integer> buildBSTFromSortedList( 
DoublyLinkedListNode<Integer> L, int length) { 
head = L; 

return buildSortedListHelper (Q , length); 

} 

// Builds a BST from the (start + l)-th to the end-th node, inclusive, in L, 
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// and returns the root. 

private static DoublyLinkedListNode<Integer> buildSortedListHelper (int start, 

int end) { 

if (start >= end) { 

return null; 

} 

int mid = start + ((end - start) / 2); 

DoublyLinkedListNode<Integer> left = buildSortedListHelper(start, mid); 

// Previous function call sets head to the successor of the maximum node in 
// the tree rooted at left. 

DoublyLinkedListNode<Integer> curr 

= new DoublyLinkedListNode<>(head.data, left, null); 
head = head.next; 

curr.next = buildSortedListHelper(mid + 1, end); 
return curr; 


The algorithms spends 0( 1) time per node, leading to an 0(n) time complexity. No 
dynamic memory allocation is required. The maximum number of call frames in the 
function call stack is fig ri], yielding an <9(log n ) space complexity. 


25.23 Convert a BST to a sorted doubly linked list 

A BST node has two references, left and right. A doubly linked list node has two 
references, previous and next. If we interpret the BST's left pointer as previous and 
the BST's right pointer as next, a BST's node can be used as a node in a doubly linked 
list. Also, the inorder traversal of a BST represents an ordered set just like a doubly 
linked list. Therefore it is natural to ask if is possible to take a BST and rewrite its 
node reference fields so that it represents a doubly linked list such that the resulting 
list represents the inorder traversal sequence of the tree. 

Design an algorithm that takes as input a BST and returns a sorted doubly linked list 
on the same elements. Your algorithm should not do any dynamic allocation. The 
original BST does not have to be preserved; use its nodes as the nodes of the resulting 
list, as shown in Figure 25.4 on the previous page. 

Hint: The tricky part is attaching the root to its subtrees. 

Solution: In the absence of the allocation constraint, the problem can be easily solved 
using a dynamic array to write nodes to a list as we perform an inorder traversal. The 
time complexity is 0(n), where n is the number of nodes, but the space complexity is 
also 0(n). 

Speaking generally, a key benefit of lists is that we can easily append one list to 
another. In particular, if we have lists for the left and right subtrees, we can easily 
splice them in with the root in 0(1) time. 

private static class HeadAndTail { 
public BSTNode<Integer> head; 
public BSTNode<Integer> tail; 

public HeadAndTail(BSTNode<Integer> head, BSTNode<Integer> tail) { 
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this. head = head; 
this .tail = tail; 

} 

} 

public static BSTNode<Integer> bstToDoublyLinkedList(BSTNode<Integer> tree) { 

return bstToDoublyLinkedListHelper(tree).head; 

} 

// Transforms a BST into a sorted doubly linked list in-place, and return the 

// head and tail of the list. 

private static HeadAndTail bstToDoublyLinkedListHelper( 

BSTNode<Integer> tree) { 

// Empty subtree. 

if (tree == null) { 

return new HeadAndTail (null, null); 

} 

// Recursively build the list from left and right subtrees. 

HeadAndTail left = bstToDoublyLinkedListHelper(tree.left); 

HeadAndTail right = bstToDoublyLinkedListHelper(tree.right); 

// Append tree to the list from left subtree. 

if (left.tail != null) { 
left.tail.right = tree; 

} 

tree.left = left.tail; 


// Append the list from right subtree to tree. 
tree.right = right.head; 
if (right.head != null) { 
right.head.left = tree; 

> 


return new HeadAndTail(left.head != null ? left.head : tree, 

right.tail != null ? right.tail : tree); 


} 


Since we do a constant amount of work per tree node, the time complexity is O(n). The 
space complexity is the maximum depth of the function call stack, i.e., 0(h), where h 
is the height of the BST. The worst-case is for a skewed tree —n activation records are 
pushed on the stack. 


25.24 Merge two BSTs 

Given two BSTs, it is straightforward to create a BST containing the union of their 
keys: traverse one, and insert its keys into the other. 

Design an algorithm that takes as input two BSTs and merges them to form a balanced 
BST. For any node, you can update its left and right subtree fields, but cannot change 
its key. See Figure 25.5 on the following page for an example. Your solution can 
dynamically allocate no more than a few bytes. 

Hint: Can you relate this problem to Problems 25.23 on the preceding page and 25.22 on Page 470? 


473 



9x391® 13 


9x205® 



9x379® 

(a) Two BSTs. 



19 9x412® 



(b) Merged BST corresponding to the two BSTs in (a). 


Figure 25.5: Example of merging two BSTs. The number in hex adjacent to each node represents its 
address in memory. 


Solution: A brute-force approach is to traverse one tree and add its keys to the second 
tree. The time complexity depends on how balanced the second tree is, and how we 
perform the insert. In the best-case, we start with the second tree being balanced, and 
preserve balance as we perform additions, yielding a time complexity of 0(n log n), 
where n is the total number of nodes. Performing the updates while reusing existing 
nodes is tricky if we do an inorder walk since the links change. However, it is fairly 
simple if we do a post-order walk, since when we visit the node, we do not need 
any information from its original left and right subtrees. Note that adding nodes 
while preserving balance is nontrivial to implement, see Solution 15.10 on Page 272 
for details. 

Looking more carefully at the brute-force approach, it is apparent that it does not 
exploit the fact that both the sources of data being merged are sorted. In particular, if 
memory was not a constraint, we could perform an inorder walk on each tree, writing 
the result of each to a sorted array. Then we could put the union of the two arrays 
into a third sorted array. Finally, we could build a balanced BST from the union array 
using recursion, e.g., using Solution 15.9 on Page 271. The time complexity would be 
0(n), but the additional space complexity is 0(n). 

It is good to remember that a list can be viewed as a tree in which each node's 
left child is empty. It is relatively simple to create a list of the same nodes as a tree 
(Solution 25.23 on Page 472). We can take these two lists and form a new list on the 
same nodes which is the union of the keys in sorted order (Solution 8.1 on Page 115). 
This new list can be viewed as a tree holding the union of the keys of the original trees. 
The time complexity is 0(n), and space complexity is 0(h ), where h is the maximum 
of the heights of the two initial trees. 

The problem with this approach is that while it meets the time and space con- 
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straints, it returns a tree that is completely unbalanced. However, we can convert 
this tree to a height-balanced one using Solution 25.22 on Page 470, completing the 
desired construction. 


public static BSTNode<Integer> mergeTwoBSTs(BSTNode<Integer> A, 

BSTNode<Integer> B) { 

A = bstToDoublyLinkedList(A); 

B = bstToDoublyLinkedList(B); 

A. left.right = null; 

B. left.right = null; 

A. left = null; 

B. left = null; 

int ALength = countLength(A); 
int BLength = countLength(B); 

return buildSortedDoublyLinkedList(mergeTwoSortedLinkedLists(A, 

ALength + BLength); 


} 


B), 


The time complexity of each stage is 0(n), and since we recycle storage, the additional 
space complexity is dominated by the time to convert a BST to a list, which is 0(h). 


25.25 The view from above © 

This is a simplified version of a problem that often comes up in computer graphics. 

You are given a set of line segments. Each segment consists of a closed interval 
of the X-axis, a color, and a height. When viewed from above, the color at point x 
on the X-axis is the color of the highest segment that includes x. This is illustrated in 
Figure 25.6. 


View from above 



0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 

Figure 25.6: Instance of the view from above problem, with patterns used to denote colors. Letters label 
input line segments. 


Write a program that takes lines segments as input, and outputs the view from above 
for these segments. You can assume no two segments whose intervals overlap have 
the same height. 

Hint: First organize the individual line segments. 
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Solution: A key observation is that, viewed from above, the color can change at 
most at an endpoint. This discretizes the problem—we only need to consider the 
endpoints, of which there are at most twice as many as line segments. 

A brute-force algorithm then is to take these endpoints, and for each endpoint, 
find the highest line segment containing it. To express the result as line segments, we 
need to sort endpoints 

Finding the highest line segment containing an endpoint has time complexity 0{n), 
where n is the number of line segments, leading to an 0(n 2 ) overall time bound. 

We can improve the time complexity by recognizing that it is grossly inefficient 
to test each endpoint against all line segments. Instead, we should track just the line 
segments that contain the endpoint being processed, and to get the color efficiently 
we should keep these line segments sorted by height 

Specifically, we scan endpoints, maintaining the set of line segments that intersect 
the current endpoint. This set is stored in a BST with the height being the key. The 
color is determined by the highest line segment. When we encounter a left endpoint, 
we add the corresponding line segment in a BST. When we encounter a right endpoint, 
we remove the corresponding line segment from the BST. 

As a concrete example, consider Figure 25.6 on the previous page. When we are 
processing J's left endpoint, the BST consists of H and I, with H appearing first (since 
it is lower than I). We add J to the BST, in between H and I. The next endpoint 
processed is J's right endpoint, at which stage we remove I from the BST. Now J is 
the highest line segment. From J's left endpoint to J's right endpoint, the view from 
above will be J. 

public static class LineSegment { 

public int left, right; // Specifies the interval. 
public int color; 
public int height; 

public LineSegment (int left, int right, int color, int height) { 
this. left = left; 
this. right = right; 
this. color = color; 
this. height = height; 

} 

} 

public static class Endpoint implements Comparable<Endpoint> { 
private boolean isLeft; 
private LineSegment line; 

public Endpoint (boolean isLeft, LineSegment line) { 
this. isLeft = isLeft; 
this . line = line ; 

} 

public int value() { return isLeft ? line.left : line.right; } 

©Override 

public int compareTo(Endpoint o) { return Integer.compare(value(), 
o.value()); } 


476 



} 


public static List<LineSegment> calculateViewFromAbove(ListcLineSegment> A) { 
List<Endpoint> sortedEndpoints = new ArrayList<>(); 
for (LineSegment a : A) { 

sortedEndpoints.add(new Endpoint(true, a)); 
sortedEndpoints.add(new Endpoint(false, a)); 

} 

Collections.sort(sortedEndpoints); 

ListcLineSegment> result = new ArrayList<>(); 

int prevXAxis = sortedEndpoints.get(®).value(); // Leftmost end point. 
LineSegment prev = null; 

TreeMapcInteger, LineSegment> activeLineSegments = new TreeMap<>(); 
for (Endpoint endpoint : sortedEndpoints) { 

if (!activeLineSegments.isEmpty() && prevXAxis != endpoint.value()) { 
if (prev == null) { // Found first segment. 
prev = new LineSegment( 

prevXAxis, endpoint.value(), 

activeLineSegments.lastEntry().getValue().color, 
activeLineSegments.lastEntry().getValue().height); 

} else { 

if (prev.height == activeLineSegments.lastEntryO.getValue().height 
&& prev.color == activeLineSegments.lastEntry().getValue().color 
&& prevXAxis == prev.right) { 
prev.right = endpoint.value(); 

} else { 

result.add(prev) ; 
prev = new LineSegment( 

prevXAxis, endpoint.value(), 

activeLineSegments.lastEntry().getValue().color, 
activeLineSegments.lastEntryO■getValue().height); 

} 

} 

} 

prevXAxis = endpoint.value(); 

if (endpoint.isLeft) { // Left end point. 

activeLineSegments.put(endpoint.line.height, endpoint.line); 

} else { // Right end point. 

activeLineSegments.remove(endpoint.line.height); 

} 

} 

// Output the remaining segment (if any). 
if (prev != null) { 
result.add(prev) ; 

} 

return result; 


The time to sort is O(n\ogn). Assuming that the BST library implementation is 
height-balanced, processing each endpoint takes <9(log n) time. Therefore, the time 
complexity is 0(n log n). 
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Variant: Solve the same problem when multiple segments may have the same height. 
Break ties arbitrarily. 

Variant: Design an efficient algorithm for computing the length of the union of a set 
of closed intervals. 

Variant: Design an efficient algorithm for computing the area of a set of rectangles 
whose sides are aligned with the X and Y axes. 

Variant: Runners R!,R 2 / • • • , race on a track of length L. Runner R; begins at an 
offset Si from the start of the track, and runs at speed Vj. Compute the set of runners 
who lead the race at some time. 

Variant: Given a set H of nonintersecting horizontal line segments in the 2D plane, 
and a set V of nonintersecting vertical line segments in the 2D plane, determine if any 
pair of line segments intersect. 


25.26 Implement regular expression matching 

A regular expression is a sequence of characters that forms a search pattern. For this 
problem we define a simple subset of a full regular expression language. We describe 
regular expressions by examples, rather than a formal syntax and semantics. 

A regular expression is a string made up of the following characters: alphanumeric, 
. (dot), * (star), A , and $. Examples of regular expressions are a, aW, aW.9, aW.9*, 
aW.*9*, A a, aW$, and A aW.9*$. Not all strings are valid regular expressions. For 
example, if A appears, it must be the first character, if $ appears, it must be the 
last character, and star must follow an alphanumeric character or dot. Beyond the 
base cases—a single alphanumeric character, dot, a single alphanumeric character 
followed by a star, dot followed by star—regular expressions are concatenations of 
shorter regular expressions. 

Now we describe what it means for a regular expression to match a string. Intu¬ 
itively, an alphanumeric character matches itself, dot matches any single character, 
and star matches zero or more occurrences of the preceding character. 

In the absence of A and $, there is no concept of an "anchor". In particular, if 
the string contains any substring matched by the regular expression, the regular 
expression matches the string itself. 

The following examples illustrate the concept of a regular expression matching a 
string. More than one substring may be matched. If a match exists, we underline a 
matched substring. 

• aW9 matches any string containing aW9 as a substring. For example, aW9, 
aW9bcW, ab8aW9, and cc2aW9raW9z are all matched by aW9, but aW8, bcd8, 
and xy are not. 

• a.9. matches any string containing a substring of length 4 whose first and third 
characters are a and 9, respectively. For example, ab9w, ac9bcW, ab8a999, and 
cc2aW9r are all matched by a.9., but az9, a989a, and bac9 are not. 

• aW*9 matches any string containing a substring beginning with a, ending with 
9, with zero or more Ws in between. For example, a9, aW9, aWW9b9cW, 
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aU9 aWW9 , ab8aWWW9W9aa, and cc2a9raW9zWW9ac are all matched by 
aW*9, but aWWU9, baX9, and aXW9Wa are not. 

• a.*9 matches any string containing a substring beginning with a ending with 
9, with zero or more characters between. For example, a9, aZ9 , aZW9b9cW, 
aU9 a9, b8aWUW9W, and cc2a9raU9z are all matched by a. *9, but 9UWaW8, 
b9aaaX, and XUq8 are not. 

• aW9* .b3 matches any string containing a substring beginning with aW, followed 
by zero or more 9s, followed by a single character, followed by b3. For example, 
ce aW999zb34 b3az, ceaW9b34, and pqaWzb38q are matched by aW9*.b3, but 
ceaW98zb34 and pqaW988b38q are not. 

If the regular expression begins with A , that indicates the match must begin at the 
start of the string. If the regular expression ends with $, the match must end at the 
end of the string. 

• A aW.9 matches strings which begin with a substring consisting of a, followed 
by W, followed by any character, followed by 9. For example, aW99zer, aWW9, 
and aWP9 GA are all matched by A aW.9, but baWx9, aW9, and aWcc90 are not. 

• aW.9$ matches strings whose last character is 9, third last character is W and 
fourth last character is a. (The second last character can be anything.) For 
example, aWW9, aWW9abc aWz9 , ba aWX9 , and abcaWP9, are all matched by 
aW.9$, but aWW99, aW, and aWcc90 are not. 

• A aW9$ is matched by aW9 and nothing else. 

Design an algorithm that takes a regular expression and a string, and checks if the 
regular expression matches the string. 

Hint: Regular expressions are defined recursively. 

Solution: The key insight is that regular expressions are defined recursively, both 
in terms of their syntax (what strings are valid regular expressions), as well as their 
semantics (when does a regular expression match a string). This suggests the use of 
recursion to do matching. 

First, some notation: s k denotes the kth suffix of string s, i.e., the string resulting 
from deleting the first k characters from s. For example, if s = aWaW9W9, then s° = 
aWaW9W9, and s 2 = aW9W9. 

Let r be a regular expression and s a string. If r starts with A , then for r to match s, 
the remainder of r must match a prefix of s. If r ends with a $, then for r to match s, 
some suffix of s must be matched by r without the trailing $. If r does not begin with 
A , or end with $, r matches s if it matches some substring of s. 

A function that checks whether a regular expression matches a string at its begin¬ 
ning has to check the following cases: 

(1.) Length-0 regular expressions. 

(2.) A regular expression starting with A or ending with $. 

(3.) A regular expression starting with a * match, e.g., a*wXY or .*Wa. 

(4.) A regular expression starting with an alphanumeric character or dot. 

Case (1.) is trivial, we just return true. Case (2.) entails a single call to the match 
function for a regular expression beginning with A , and some checking logic for a 
regular expression ending with $. Case (3.) is handled by a traversal down the string 
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checking that the prefix of the string thus far matches the alphanumeric character or 
dot until some suffix is matched by the remainder of the regular expression. Each 
suffix check is a call to the match function. Case (4.) involves examining a character, 
possibly followed by a call to the match function. 

As an example, consider the regular expression ab.c*d. To check if it matches 
s = caeabbedeabacccde, we iterate over the string. Since s[0] = c, we cannot match 
the regular expression at the start of s. Next we try s 1 . Since s[l] = a, we recursively 
continue checking with s 1 and b.c*d. However s[ 2] ^ b, so we return false from this 
call. Continuing, s 2 is immediately eliminated. With s 3 , since s[3] = a, we continue 
recursively checking with s 4 and b.c*d. Since s[4] = b, we continue checking with 
.c*d. Since dot matches any single character, it matches s[5], so we continue checking 
with c*d. Since s[ 6] = e, the only prefix of s 6 which matches c*d is the empty one. 
However, when we continue checking with d, since s[6] ^ d, we return false for this 
call. Skipping some unsuccessful checks, we get to s 9 . Similar to before, we continue 
to s 12 , and c*d. The string beginning at offset 12 matches c* with prefixes of length 
0,1,2,3. After the first three, we do not match with the remaining d. However, after 
the prefix ccc, the following string does end in d, so we return true. 


public static boolean isMatch(String regex, String s) { 
// Case (2.): regex starts with ’ A ’. 
if (regex.charAt(®) == ’ A ’) { 

return isMatchHere(regex.substring(1), s); 

} 

for (int i = ®; i <= s.lengthO; ++i) { 
if (isMatchHere(regex, s.substring(i))) { 

return true; 

} 

} 

return false; 


private static boolean isMatchHere(String regex, String s) { 
if (regex.isEmpty()) { 

// Case (1.): Empty regex matches all strings. 

return true; 

} 

if ("$".equals(regex)) { 

// Case (2): Reach the end of regex, and last char is ’S’. 
return s.isEmpty(); 

} 

if (regex.length() >= 2 && regex.charAt(1) == ’*’) { 

// Case (3.): A ’*’ match. 

// Iterate through s, checking ’*’ condition, if ’*’ condition holds, 
// performs the remaining checks . 

for (int i = ®; i < s.lengthO && (regex. charAt (®) == 

|| regex.charAt(®) == s.charAt(i)) ; 

++i) { 

if (isMatchHere(regex.substring(2), s.substring(i + 1))) { 

return true; 

} 
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} 

// See matches zero character in s. 

return isMatchHere(regex.substring(2), s); 

} 

// Case (4.): regex begins with single character match. 
return Is.isEmptyO 

&& (regex.charAt(®) == ’.’ || regex.charAt(®) == s.charAt(®)) 

&& isMatchHere(regex.substring(1), s.substring(1)); 


Let C(x, k) be k copies of the string x concatenated together. For the regular expression 
C(a.*,k) and string C(ab,k- 1) the algorithm presented above takes time exponential 
in k. We cannot give a more precise bound on the time complexity. 

Variant: Solve the same problem for regular expressions without the A and $ opera¬ 
tors. 


25.27 Synthesize an expression & 

Consider an expression of the form (3 O 1 O 4 O 1 O 5), where each O is an operator, 
e.g., +, —, X, +. The expression takes different values based on what the operators 
are. Some examples are 14 (if all operators are +), 60 (if all operators are x), and 22 
(3 - 1 + 4 x 1 x 5). 

Determining an operator assignment such that the resulting expression takes a 
specified value is, in general, a difficult computational problem. For example, sup¬ 
pose the operators are + and and we want to know whether we can select each 
O such that the resulting expression evaluates to 0. The problem of partitioning a 
set of integers into two subsets which sum up to the same value, which is a famous 
NP-complete problem, directly reduces to our problem. 

Write a program that takes an array of digits and a target value, and returns true if it 
is possible to intersperse multiplies (x) and adds (+) with the digits of the array such 
that the resulting expression evaluates to the target value. For example, if the array is 
(1,2,3,2,5,3,7,8,5,9) and the target value is 995, then the target value can be realized 
by the expression 123 + 2 + 5x3x7 + 85x9, so your program should return true. 

Hint: Build the assignment incrementally. 

Solution: Let A be the array of digits and k the target value. We want to intersperse 
X and + operations among these digits in such a way that the resulting expression 
equals k. 

For each pair of characters, (A[i], A[i + 1]), we can choose to insert a X, a +, or no 
operator. If the length of A is n, the number of such locations is n — 1, implying we can 
encode the choice with an array of length n — 1. Each entry is one of three values— X, 
+, and(which indicates no operator is added at that location). There are exactly 3" _1 
such arrays, so a brute-force solution is to systematically enumerate all arrays. For 
each enumerated array, we compute the resulting expression, and return as soon as 
we evaluate to k. The time complexity is 0(n X 3”), since each expression takes time 
0(n) to evaluate. 
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To improve runtime, we use a more focused enumeration. Specifically, the first 
operator can appear after the first, second, third, etc. digit, and it can be a + or X. For 
+ to be a possibility, it must be that the sum of the value given by the initial operator 
assignment plus the value encoded by the remaining digits is greater than or equal to 
the target value. This is because the maximum value is achieved when there are no 
operators. For example, for (1,2,3,4,5), we can never achieve a target value of 1107 
if the first operator is a + placed after the 3 (since 123 + 45 < 1107). This gives us a 
heuristic for pruning the search. 

public static boolean expressionSynthesis (Listdnteger > digits, int target) { 

List<Character> operators = new ArrayList<>(); 

Listdnteger> operands = new ArrayList <>() ; 

return directedExpressionSynthesis(digits, target, ®, ®, operands, 


private static boolean directedExpressionSynthesis( 

Listdnteger> digits, int target, int currentTerm, int offset, 
Listdnteger> operands, List<Character> operators) { 
currentTerm = currentTerm * 1® + digits.get(offset); 
if (offset == digits.size() - 1) { 
operands.add(currentTerm); 

if (evaluate(operands, operators) == target) { // Found a match. 

return true; 

} 

operands.remove(operands.size() - 1); 

return false; 


// No operator. 

if (directedExpressionSynthesis(digits, target, currentTerm, offset + 1, 

operands, operators)) { 

return true; 

} 

// Tries multiplication operator 
operands.add(currentTerm); 
operators.add(’*’); 

if (directedExpressionSynthesis(digits, target, ®, offset + 1, operands, 

operators)) { 

return true; 

} 

operators.remove(operators.size() - 1); 
operands.remove(operands.size() - 1); 

// Tries addition operator ’+’. 
operands.add(currentTerm); 

if (target - evaluate(operands, operators) 

<= remaininglnt(digits, offset + 1)) { 
operators.add(’+’); 

if (directedExpressionSynthesis(digits, target, ®, offset + 1, operands, 

operators)) { 


return true; 

} 

operators.remove(operators.size() - 1) ; 

} 

operands.remove(operands.size() - 1); 


482 



return false; 


// Calculates the int represented by digits.subList(idx, digits.size()). 
private static int remaininglnt(Listdnteger> digits, int idx) { 
int val = ®; 

for (int i = idx; i < digits.size(); ++i) { 
val = val * 1® + digits.get (i) ; 

} 

return val; 


private static int evaluate(List<Integer> operands, 

List<Character> operators) { 

Dequeclnteger> intermediateOperands = new LinkedList<>(); 
int operandldx = ®; 

intermediateOperands.addFirst(operands.get(operandIdx++)); 

// Evaluates ’*’ first. 
for (char oper : operators) { 
if (oper == ’*’) { 

intermediateOperands.addFirst(intermediateOperands.removeFirst() 

* operands.get(operandIdx++)); 

} else { 

intermediateOperands.addFirst(operands.get(operandldx++)); 

} 

} 

// Evaluates ’+’ second. 
int sum = ®; 

while (! intermediateOperands . isEmptyO) { 
sum += intermediateOperands.removeFirst(); 

> 

return sum; 


Despite the heuristics helping in some cases, we cannot prove a better bound for the 
worst-case time complexity than the original 0(n3 n ). 


25.28 Count inversions 

Let A be an array of integers. Call the pair of indices (i, j) inverted if i < j and 
A[i] > A[j]. For example, if A = (4,1,2,3), then the pair of indices (0,3) is inverted. 
Intuitively, the number of inverted pairs in an array is a measure of how unsorted it 
is. 


Design an efficient algorithm that takes an array of integers and returns the number 
of inverted pairs of indices. 

Hint: Let A and B be arrays. How would you count the number of inversions where one element 
is from A and the other from B in the array consisting of elements from A and B? 

Solution: The brute-force algorithm examines all pairs of indices (z,;), where i < j. 
has an 0(n 2 ) complexity, where n is the length of the array. 
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One way to recognize that the brute-force algorithm is inefficient is to consider 
how we would check if an array is sorted. We would not test for every element if 
all subsequent elements are greater than or equal to it—we just test the next element, 
since greater than or equal is transitive. 

This suggests the use of sorting to speed up counting the number of inverted pairs. 
In particular, if we sort the second half of the array, then to see how many inversions 
exist with an element in the first half, we could do a binary search for that element 
in the second half. The elements before its location in the second half are the ones 
inverted with respect to it. 

Elaborating, suppose we have counted the number of inversions in the left half L 
and the right half R of A. What are the inversions that remain to be counted? Sorting 
L and R makes it possible to efficiently obtain this number. For any ( i , j) pair where i 
is an index in L and j is an index in R, if L[i ] > R[j], then for all j' < j we must have 

nn > Rin 

For example, if A = (3,6,4,2,5,1), then L = (3,6,4), and R = (2,5,1). After 
sorting, L = (3,4,6), and R = (1,2,5). The inversion counts for L and R are 1 and 2, 
respectively. When merging to form their sorted union, since 1 < 3, we know 4,6 are 
also inverted with respect to 1, so we add |L| - 0 = 3 to the inversion count. Next we 
process 2. Since 2 < 3, we know 4,6 are also inverted with respect to 2, so we add 
\L\ - 0 = 3 to the inversion count. Next we process 3. Since 5 > 3,3 does not add any 
more inversions. Next we process 4. Since 5 > 4,4 does not add any more inversions. 
Next we process 5. Since 5 < 6, we add |L| - 2 (which is the index of 6) = 1 to the 
inversion count. In all we add 3 + 3 + 1 = 7 to inversion counts for L and R (which 
were 1 and 2, respectively) to get the total number of inversions, 10. 


public static int countlnversions(Listclnteger> A) { 
return countSubarraylnversions (A, ®, A.sizeO); 

} 

// Return the number of inversions in A.subList(start, end). 
private static int countSubarraylnversions(Listclnteger> A, int start, 

int end) { 

if (end - start <= 1) { 

return ®; 

} 

int mid = start + ((end - start) / 2); 
return countSubarraylnversions(A, start, mid) 

+ countSubarraylnversions(A, mid, end) 

+ mergeSortAndCountlnversionsAcrossSubarrays(A, start, mid, end); 

} 

// Merge two sorted sublists AsubList(start, mid) and A.subList(mid, end) 

// into A . subList(start, end) and return the number of inversions across 
// A . subList(start, mid) and A.subList(mid , end). 

private static int mergeSortAndCountlnversionsAcrossSubarrays(List<Integer> A, 

int start , 
int mid, 
int end) { 

List<Integer> sortedA = new ArrayList<>(); 

int leftStart = start, rightStart = mid, inversionCount = ®; 
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while (leftStart < mid && rightStart < end) { 

if (Integer.compare(A.get(leftStart), A.get(rightStart)) <= Q) { 
sortedA.add(A.get(leftStart++)); 

} else { 

// A.subList(leftStart, mid) are the inversions of A[rightStart]. 
inversionCount += mid - leftStart; 
sortedA.add(A.get(rightStart++)); 

} 

} 

sortedA.addAll(A.subList(leftStart, mid)); 
sortedA.addAll(A.subList(rightStart, end)); 

// Updates A with sortedA. 
for (Integer t : sortedA) { 

A.set(start++, t); 

} 

return inversionCount; 


The time complexity satisfies T(n) = O(n)+2T(n— 1), which solves to 0(n log n), where 
n is the length of the array. 

Variant: Runners numbered from 0 to n - 1 race on a straight one-way road to a 
common finish line. The runners have different (constant) speeds and start at different 
distances from the finish line. Specifically, Runner i has a speed s,- and begins at a 
distance d x from the finish line. Each runner stops at the finish line, and the race ends 
when all runners have reached the finish line. How many times does one runner pass 
another? 


25.29 Draw the skyline © 

A number of buildings are visible from a point. A building appears as a rectangle, 
with the bottom of each building lying on a fixed horizontal line. A building is 
specified using its left and right coordinates, and its height. One building may partly 
obstruct another, as shown in Figure 25.7(a) on the following page. The skyline is the 
list of coordinates and corresponding heights of what is visible. 

For example, the skyline corresponding to the buildings in Figure 25.7(a) on the 
next page is given in Figure 25.7(b) on the following page. (The patterned rectangles 
within the skyline are used to describe Problem 18.8 on Page 347; they are not relevant 
to the current problem.) 

Design an efficient algorithm for computing the skyline. 

Hint: Think of an efficient way of merging skylines. 

Solution: The simplest solution is to compute the skyline incrementally. For one 
building, the skyline is trivial. Suppose we know the skyline for some buildings, and 
need to compute the skyline when another building is added. Let the new building's 
left and right coordinates be L and R, and its height H. To add it, we iterate through 
the existing skyline from left to right to see where L should be added. Then we move 
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(a) A set of buildings. 
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(b) Skyline for the buildings in (a). 


Figure 25.7: Buildings, their skyline, and the largest contained rectangle. The text label identifying the 
building is just below and to the right of its upper left-hand corner. 


through the existing skyline until we pass R, increasing any heights that are less than 
H to H. 

This algorithm is simple, but has 0(n 2 ) complexity if there are n buildings, since 
adding the nth building may entail 0(n) comparisons. The key to improving efficiency 
is the observation that it takes linear time to merge two skylines (if they are represented 
in left-to-right order), which is the same as the time to merge a single skyline, but gets 
much more done. 

Now it is clear what the textbook solution is: use divide-and-conquer to com¬ 
pute skylines for one half of the buildings, the other half of the buildings, and then 
merge the results. The merge is similar to the procedure for adding a single building, 
described above, and can be performed in 0(n) time. We iterate through the two sky¬ 
lines together from left-to-right, matching their X coordinates, and updating heights 
appropriately. Our website has a link to a program based on the algorithm above. 
The time complexity T(n) satisfies the recurrence T(n) = 2T(n/2) + 0(n), where the 
latter term comes from the merge step. This solves to T(n) = 0(n log n). 

As an example, consider merging the skyline for Buildings A,B,C,D with the 
skyline for Buildings E, F, G, H. From 0 to 7, the skyline for A, B, C, D determines 
the height, because these are the only buildings present. At 7, since the skyline for 
A, B, C, D is taller, than that for E, F, G, H, we use its height, 4, going forward. At 8, 
since the skyline for E, F, G, H is taller, so we use its height, 3, going forward. After 9, 
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the skyline for E, F, G, H determines the height, because these are the only buildings 
going forward. 

Here is an alternative to the textbook solution that is hugely simpler, and, except 
for degenerate inputs, performs much better. It is based on "digitizing" the problem. 
Let's say the left-most coordinate for any building is l and the right-most coordinate 
for any building is r. Then the final skyline will start at l and end at r. We draw the 
the skyline building by building as follows. We represent the skyline with an array, 
where the ith element of the array hold the height of coordinate i of the current skyline. 
This array is initialized to Os. For each building, we iterate over the coordinates, and 
update the skyline—if the current skyline's height at coordinate i is less than the 
height of the building, we update the skyline at i to the building's height. 

public static class Rectangle { 
public int left, right, height; 

public Rectangle(int left, int right, int height) { 
this.left = left; 
this.right = right; 
this.height = height; 

} 

} 

public static List<Rectangle> drawingSkylines(List<Rectangle> buildings) { 
int minLeft = Integer.MAX.VALUE, maxRight = Integer.MINJVALUE; 
for (Rectangle building : buildings) { 

minLeft = Math.min(minLeft, building.left); 
maxRight = Math.max(maxRight, building.right); 

} 

List<Integer> heights 

= new ArrayList<>(Collections.nCopies(maxRight - minLeft + 1, ®)); 
for (Rectangle building : buildings) { 

for (int i = building.left; i <= building.right; ++i) { 
heights.set(i - minLeft, 

Math.max(heights.get(i - minLeft), building.height)); 

} 

} 

List<Rectangle> result = new ArrayList<>(); 
int left = ®; 

for (int i = 1; i < heights. size () ; ++i) { 
if (heights.get(i) != heights.get(i - 1)) { 

result.add( 

new Rectangle(left + minLeft, i - 1 + minLeft, heights.get(i - 1))); 
left = i; 

} 

} 

result.add(new Rectangle(left + minLeft, maxRight, 

heights.get(heights.size() - 1))); 

return result; 


The time complexity is 0(nW), where W is the width of the widest building. In theory, 
W could be very large, making this approach much worse than the textbook solution. 


487 



In practice, W will be a constant, and the digitized solution will be much faster. It is 
also vastly simpler to code and to understand. 

Variant: Solve the skyline problem when each building has the shape of an isosceles 
triangle with a 90 degree angle at its apex. 


25.30 Measure with defective jugs 

You have three measuring jugs. A, B, and C. The measuring marks have worn out, 
making it impossible to measure exact volumes. Specifically, each time you measure 
with A, all you can be sure of is that you have a volume that is in the range [230,240] 
mL. (The next time you use A, you may get a different volume—all that you know 
with certainty is that the quantity will be in [230,240] mL.) Jugs B and C can be used 
to measure a volume in [290,310] mL and in [500,515] mL, respectively. Your recipe 
for chocolate chip cookies calls for at least 2100 mL and no more than 2300 mL of 
milk. 

Write a program that determines if there exists a sequence of steps by which the 
required amount of milk can be obtained using the worn-out jugs. The milk is 
being added to a large mixing bowl, and hence cannot be removed from the bowl. 
Furthermore, it is not possible to pour one jug's contents into another. Your scheme 
should always work, i.e., return between 2100 and 2300 mL of milk, independent 
of how much is chosen in each individual step, as long as that quantity satisfies the 
given constraints. 

Hint: Solve the n jugs case. 

Solution: It is natural to solve this problem using recursion—if we use jug A for the 
last step, we need to correctly measure a volume of milk that is at least 2100 - 230 = 
1870 mL—the last measurement may be as little as 230 mL, and anything less than 
1870 mL runs the risk of being too little. Similarly, the volume must be at most 
2300 - 240 = 2060 mL. The volume is not achievable if it is not achievable with any of 
the three jugs as ending points. We cache intermediate computations to reduce the 
number of recursive calls. 

In the following code, we implement a general purpose function which finds the 
feasibility among n jugs. 

public static class Jug { 
public int low, high; 

public Jug () {} 

public Jug (int low, int high) { 
this. low = low; 
this. high = high; 

} 

} 

private static class VolumeRange { 
public Integer low; 
public Integer high; 
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public VolumeRange(Integer low, Integer high) { 
this. low = low; 
this. high = high; 

} 

©Override 

public boolean equals(Object obj) { 

if (obj == null || !(obj instanceof VolumeRange)) { 

return false; 

} 

if (this == obj) { 
return true; 

} 

VolumeRange vr = (VolumeRange)obj; 

return low . equals (vr . low) <&& high . equals (vr . high) ; 

} 

©Override 

public int hashCode() { return Objects.hash(low, high); } 

} 

public static boolean checkFeasible(List<Jug> jugs, int L, int H) { 
Set<VolumeRange> cache = new HashSet<>(); 
return checkFeasibleHelper(jugs, L, H, cache); 

} 

private static boolean checkFeasibleHelper(List<Jug> jugs, int L, int H, 

Set<VolumeRange> c) { 

if (L > H || c.contains (new VolumeRange(L, H)) || (L < 0 && H < ®)) { 

return false; 

} 

// Checks the volume for each jug to see if it is possible. 
for (Jug j : jugs) { 

if ((L <= j . low &<& j .high <= H) || // Base case: j is contained in [L, H]. 
checkFeasibleHelper(jugs, L - j.low, H - j.high, c)) { 
return true; 

} 

} 

c.add(new VolumeRange(L, H)); // Marks this as impossible. 

return false; 


The time complexity is 0((L + 1)(H + 1 )ri). The time directly spent within each call to 
CheckFeasibleHelper, except for the recursive calls, is 0(n), and because of the cache, 
there are at most (L + 1 )(H + 1) calls to CheckFeasibleHelper. The space complexity 
is 0{(L + 1 )(H + 1)), which is the upper bound on the size of the cache. 

Note that it is possible to formulate this problem using Integer Linear Program¬ 
ming (ILP). However, typically interviewers will not be satisfied with a reduction to 
ILP since such a solution does not demonstrate any programming skills. 

Variant: Suppose Jug i can be used to measure any quantity in [Z;,u,] exactly. Deter¬ 
mine if it is possible to measure a quantity of milk between L and U. 
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25.31 Compute the maximum subarray sum in a circular array 


Finding the maximum subarray sum in an array can be solved in linear time, as 
described on Page 304. However, if the given array A is circular, which means the 
first and last elements of the array are to be treated as being adjacent to each other, the 
algorithm yields suboptimum solutions. For example, if A is the array in Figure 17.2 
on Page 305, the maximum subarray sum starts at index 7 and ends at index 3, but 
the algorithm described on Page 305 returns the subarray from index 0 to index 3. 

Given a circular array A, compute its maximum subarray sum in 0(n) time, where n 
is the length of A. Can you devise an algorithm that takes 0(n) time and 0(1) space? 

Hint: The maximum subarray may or may not wrap around. 

Solution: First recall the standard algorithm for the conventional maximum subarray 
sum problem. This proceeds by computing the maximum subarray sum S[i] when 
the subarray ends at i, which is max(S[z - 1] + A[i\,A[i]). Its time complexity is 0(n), 
where n is the length of the array, and space complexity is 0(1). 

One approach for the maximum circular subarray is to break the problem into two 
separate instances. The first instance is the noncircular one, and is solved as described 
above. 

The second instance entails looking for the maximum subarray that cycles around. 
Naively, this entails finding the maximum subarray that starts at index 0, the max¬ 
imum subarray ending at index n - 1, and adding their sums. However, these two 
subarrays may overlap, and simply subtracting out the overlap does not always give 
the right result (consider the array (10, -4,5, -4,10)). 

Instead, we compute for each i the maximum subarray sum S z for the subarray 
that starts at 0 and ends at or before i, and the maximum subarray E z for the subarray 
that starts after i and ends at the last element. Then the maximum subarray sum for 
a subarray that cycles around is the maximum over all i of S z + E z . 

public static int maxSubarraySumlnCircular(List<Integer> A) { 

return Math.max(findMaxSubarray(A), findCircularMaxSubarray(A)); 

} 

// Calculates the non-circular solution. 
private static int findMaxSubarray(List<Integer> A) { 
int maximumTill = ®, maximum = ®; 
for (Integer a : A) { 

maximumTill = Math.max(a, a + maximumTill); 
maximum = Math.max(maximum, maximumTill); 

} 

return maximum; 

} 

// Calculates the solution which is circular . 

private static int findCircularMaxSubarray(Listdnteger> A) { 

// Maximum subarray sum starts at index ® and ends at or before index i. 

ArrayList<Integer> maximumBegin = new ArrayList<>(); 

int sum = A.get(®); 

maximumBegin.add(sum); 

for (int i = 1; i < A.sizeQ; ++i) { 
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sum += A.get (i) ; 
maximumBegin.add( 

Math.max(maximumBegin.get(maximumBegin.size() - 1), sum)); 

} 

// Maximum subarray sum starts at index i + 1 and ends at the last element. 
List<Integer> maximumEnd 

= new ArrayList<>(Collections.nCopies(A.size(), ®)); 
sum = ®; 

for (int i = A.sizeO - 2; i >= ®; --i) { 
sum += A.get(i + 1); 

maximumEnd.set(i, Math.max(maximumEnd.get(i + 1), sum)); 

} 

// Calculates the maximum subarray which is circular. 
int circularMax = ®; 

for (int i = ®; i < A.sizeO; ++i) { 
circularMax 

= Math.max(circularMax, maximumBegin.get(i) + maximumEnd.get(i)); 

} 

return circularMax; 


The time complexity and space complexity are both 0(n). 

Alternatively, the maximum subarray that cycles around can be determined by 
computing the minimum subarray—the remaining elements yield a subarray that 
cycles around. (One or both of the first and last elements may not be included in this 
subarray, but that is fine.) This approach uses 0(1) space and 0(n) time; code for it is 
given below. 


private interface IntegerComparator { 

Integer compare(Integer ol, Integer o2); 

} 

private static class MaxComparator implements IntegerComparator { 
©Override 

public Integer compare(Integer ol, Integer o2) { 
return ol > o2 ? ol : o2; 

} 

} 

private static class MinComparator implements IntegerComparator { 
©Override 

public Integer compare(Integer ol, Integer o2) { 
return ol > o2 ? o2 : ol; 

} 

} 

public static int maxSubarraySumlnCircular(Listdnteger> A) { 

// Finds the max in non-circular case and circular case. 
int accumulate = ®; 
for (int a : A) { 
accumulate += a; 

} 

return Math.max( 

findOptimumSubarrayUsingComp(A, new MaxComparator()), 
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accumulate - findOptimumSubarrayUsingComp(A, new MinComparator())); 


private static int findOptimumSubarrayUsingComp(List<Integer> A, 

IntegerComparator comp) { 

int till = ®, overall = ©; 
for (int a : A) { 

till = comp.compare(a, a + till); 
overall = comp.compare(overall, till); 

} 

return overall; 


25.32 Determine the critical height ®> 

You need to test the design of a protective case. Specifically, the case can protect 
the enclosed device from a fall from up to some number of floors, and you want to 
determine what that number of floors is. You can assume the following: 

• All cases have identical physical properties. In particular, if one breaks when 
falling from a particular level, all of them will break when falling from that 
level. 

• A case that survives a fall can be used again, and a broken case must be dis¬ 
carded. 

• If a case breaks when dropped, then it would break if dropped from a higher 
floor, and if a case survives a fall, then it would survive a shorter fall. 

It is not ruled out that the first-floor windows break eggs, nor is it ruled out that eggs 
can survive the 36th-floor windows. 

You know that there exists a floor such that the case will break if it is dropped from 
any floor at or above that floor, will remain intact if dropped from a lower floor. The 
ground floor is numbered zero, and it is given that the case will not break if dropped 
from the ground floor. 

An additional constraint is that you can perform only a fixed number of drops 
before the building supervisor stops you. 

Note that if we have a single case and are allowed only 5 drops, then the highest 
we can measure to is 5 floors, testing from 1, 2, 3, 4, and 5. We cannot skip a floor, 
since the case may break immediately after the skipped floor, and we would have no 
way to know if the critical floor was the last one tested or a skipped floor. If the case 
does not break, we know it is able to last to a fifth floor drop. 

If we have two cases and are allowed 5 drops, we can do better. For example, we 
could test by dropping from floors 2,4,6, 8, 9. If a case breaks on the first four drops, 
we have narrowed the critical floor to that floor or the one below it. We can test the 
one below it with the second case. If the case breaks on the fifth drop, we know the 
critical floor is 9. If the case does not break, we know it is able to last to a ninth floor 
drop. Clearly having two cases is better than one. 

Given c cases and a maximum of d allowable drops, what is the maximum number of 
floors that you can test in the worst-case? 
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Hint: Write a recurrence relation. 


Solution: Let F(c,d) be the maximum number of floors we can test with c identical 
cases and at most d drops. We know that F(l,d) = d. Suppose we know the value of 
F(i, j) for all i < c and j < d. 

If we are given c + 1 cases and d drops we can start at floor F(c,d- 1) + 1 and drop 
a case. If the case breaks, then we can use the remaining c cases and d — 1 drops to 
determine the floor exactly, since it must be in the range [1, F(c, d- 1)]. If the case did 
not break, we proceed to floor F(c, d - 1) + 1 + F(c + 1 ,d— 1). 

Therefore, F satisfies the recurrence 

F(c + 1 ,d) = F(c,d- 1) + 1 + F(c + \,d - 1). 

We can compute F using DP as below: 

public static int getHeight (int cases, int drops) { 

List<List<Integer» F = new ArrayList<>(cases + 1) ; 
for (int i = ®; i < cases + 1; ++i) { 

F.add(new ArrayList(Collections.nCopies(drops + 1, -1))); 

} 

return getHeightHelper(cases, drops, F); 

} 

private static int getHeightHelper (int cases, int drops, 

List<List<Integer>> F) { 

if (cases == ® || drops == ®) { 
return ®; 

} else if (cases == 1) { 
return drops; 

} else { 

if (F .get(cases).get(drops) == -1) { 

F.get(cases).set(drops, getHeightHelper(cases, drops - 1, F) 

+ getHeightHelper(cases - 1, drops - 1, F) 

+ l); 

} 

return F.get(cases).get(drops); 

} 

} 


The time and space complexity are 0((c + l)(d + 1)). 

Variant: Solve the same problem with 0(c) space. 

Variant: How would you compute the minimum number of drops needed to find the 
breaking point from 1 to F floors using c cases? 

Variant: Men numbered from 1 to n are arranged in a circle in clockwise order. Every 
kth. man is removed, until only one man remains. What is the number of the last 
man? 


25.33 Find the maximum 2D subarray 

The following problem has applications to image processing. 
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Let A be an n X m Boolean 2D array. Design efficient algorithms for the following two 
problems: 

• What is the largest 2D subarray containing only Is? 

• What is the largest square 2D subarray containing only Is? 

What are the time and space complexities of your algorithms as a function of n and 
ml 

Hint: How would you efficiently check if A[i: i + a][j : j + b] satisfies the constraints, assuming 
you have already performed similar checks? 

Solution: A brute-force approach is to examine all 2D subarrays. Since a 2D subarray 
is characterized by two diagonally opposite comers the total number of such arrays is 
0(m 2 n 2 ). Each 2D subarray can be checked by examining the corresponding entries, so 
the overall complexity is 0(m 3 n 3 ). This can be easily reduced to 0(m 2 n 2 ) by processing 
2D subarrays by size, and reusing results—the 2D subarray A[i : i + a][j : j + b] is 
feasible if and only if the 2D subarrays A [i: i+a-l][j : j+b—l],A[i+a : i+a][j : j+b- 1], 
A[i : i + a - 1 ][j + b : j + b], and A[i + a : i + a][j + b : j + b] are feasible. This is an 
(9(1) time operation, assuming that feasibility of the smaller 2D subarrays has already 
been computed and stored. (Note that this solution requires 0(m 2 n 2 ) storage.) 

The following approach lowers the time and space complexity. For each feasible 
entry A[i][j] we record (/z/ ; , where h it] is the largest L such that all the entries in 
A[i : i + L - 1 ][/ : /] are feasible, and iv ir j is the largest L such that all the entries in 
A[i : i][j : ; + L - 1] are feasible. This computation can be performed in 0(mn) time, 
and requires 0(mn) storage. 

Now for each feasible entry A[i][j] we calculate the largest 2D subarray that has 
A [i][j] as its bottom-left corner. We do this by processing each entry in A [i: i+h it j-l][j : 
j]. As we iterate through the entries in vertical order, we update w to the smallest w it] 
amongst the entries processed so far. The largest 2D subarray that has A[i][j] as its 
bottom-left corner and A[i'][/] as its top-left corner has area (i f - i + 1 )zv. We track the 
largest 2D subarray seen so far across all A[i][j\ processed. 

private static class MaxHW { 
public int h, w; 

public MaxHW(int h, int w) { 
this.h = h; 
this.w = w; 

} 

} 

public static int maxRectangleSubmatrix(ListcList<Boolean» A) { 

// DP table stores (h , w) for each (i, j). 

MaxHW[][] table = new MaxHW[A.size()][A.get(®).size()]; 

for (int i = A.sizeO - 1; i >= ®; --i) { 

for (int j = A.get(i).size() - 1; j >= ®; --j) { 

// Find the largest h such that (i, j) to (i + h - 1, j) are feasible. 

// Find the largest w such that (i, j) to (i, j + w - 1) are feasible. 
table[i][j] 

= A.get(i) .get(j) 

? new MaxHW( 
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} 


i + 1 < A.sizeO ? table[i + l][j].h +1:1, 
j + 1 < A.get(i).size() ? table[i][j + 1].w +1:1) 
: new MaxHW(®, ®) ; 


} 

int maxRectArea = ®; 

for (int i = ®; i < A.sizeO; ++i) { 

for (int j = ®; j < A. get (i) . size () ; ++j) { 

// Process (i, j) if it is feasible and is possible to update 
// maxRectArea. 

if (A.get(i).get(j) && table[i][j].w * table[i][j].h > maxRectArea) { 
int minWidth = Integer.MAX.VALUE; 
for (int a = ®; a < table[i][j].h; ++a) { 

minWidth = Math.min(minWidth, table[i + a][j].w); 
maxRectArea = Math.max(maxRectArea, minWidth * (a + 1)); 

} 

} 

} 

} 

return maxRectArea; 


The time complexity per A[i][j] is proportional to the number of rows, i.e., 0(n), 
yielding an overall time complexity of 0(mn 2 ) r and space complexity of 0(mn). 

If we are looking for the largest feasible square region, we can improve the com¬ 
plexity as follows—we compute the w^j) values as before. Suppose we know the 

length s of the largest square region that has A[i + l][j + 1] as its bottom-left corner. 
Then the length of the side of the largest square with A[i][j\ as its bottom-left corner is 
at most s +1, which occurs if and only if h- h j > s +1 and w it j > s +1. The general expres¬ 
sion for the length is min(s + 1 Note that this is an 0(1) time computation. 

In total, the run time is 0(mn), a factor of n better than before. 

The calculations above can be sped up by intelligent pruning. For example, if we 
already have a feasible 2D subarray of dimensions H X W, there is no reason to process 
an entry A[i][j] for which h if} < H and w if j < W. 

public static class MaxHW { 
public int h, w; 

public MaxHW(int h, int w) { 
this.h = h; 
this.w = w; 

} 

} 

public static int maxSquareSubmatrix(List<List<Boolean» A) { 

// DP table stores (h, w) for each (i, j). 

List<List<MaxHW>> table = new ArrayList<>(A.size()); 
for (int i = ®; i < A.sizeO; ++i) { 
table.add( 

new ArrayList(Collections.nCopies(A.get(i).size(), new MaxHW(®, ®)))); 

} 

for (int i = A.sizeO - 1; i >= ®; --i) { 
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for (int j = A.get(i).size() - 1; j >= ®; --j) { 

// Find the largest h such that (i , j) to (i + h - 1, j) are feasible. 

// Find the largest w such that (i , j) to (i, j + w - 1) are feasible. 
table.get(i) . set ( 

j, A.get(i) .get(j) 

? new MaxHW( 

i + 1 < A.sizeO ? table.get(i + l).get(j).h + 1:1, 
j + 1 < A.get(i).size() ? table.get(i) . get(j + 1).w + 1 
: 1) 

: new MaxHW(®, ®)); 

} 

} 

// A table stores the length of the largest square for each (i, j). 

List<List<Integer>> s = new ArrayList<>(A.size()); 
for (int i = ®; i < A.sizeO; ++i) { 

s.add(new ArrayList(Collections.nCopies(A.get(i).size(), ®))); 

} 

int maxSquareArea = ®; 

for (int i = A.sizeO - 1; i >= ®; --i) { 

for (int j = A.get(i).size() - 1; j >= ®; --j) { 

int side = Math . min(table . get (i) . get (j ) .h, table . get (i) . get (j ) .w) ; 
if (A.get(i).get ( j)) { 

// Get the length of largest square with bottom-left corner (i, j). 
if (i + 1 < A.sizeO && 3 + 1 < A.get(i + l).size()) { 
side = Math.min(s.get(i + l).get(j + 1) + 1, side); 

} 

s.get(i).set (j , side) ; 

maxSquareArea = Math.max(maxSquareArea , side * side); 

} 

} 

} 

return maxSquareArea; 


The largest 2D subarray can be found in 0(nm) time using a qualitatively different 
approach. Essentially, we reduce our problem to n instances of the largest rectangle 
under the skyline problem described in Problem 18.8 on Page 347. First, for each 
A[i][j] we determine the largest h if j such that A[i : i + h if j — 1 ][j : j] is feasible. (If 
A[i][j] = 0 then hj f j = 0.) Then for each of the n rows, starting with the topmost one, 
we compute the largest 2D subarray whose bottom edge is on that row in time 0(m), 
using Solution 18.8 on Page 347. This computation can be performed in time 0(n) 
once the h^j values have been computed. The final solution is the maximum of the n 
instances. 

The time complexity for each row is 0(m) for computing the h values, assuming we 
record the h values for the previous row, and 0(m) for computing the largest rectangle 
under the skyline, i.e., 0{m) in total per row. Therefore the total time complexity is 
0(mn). The additional space complexity is 0(m) —this is the space for recording the h 
values and running the largest rectangle under the skyline computation. 

public static int maxRectangleSubmatrix(ListcList<Boolean» A) { 

List<Integer> table 

= new ArrayList<>(Collections.nCopies (A .get (®) .size () , ®)); 
int maxRectArea = ®; 
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// Find the maximum among all instances of the largest rectangle. 
for (int i = ®; i < A.sizeO; ++i) { 

for (int j = ®; j < A.get(i).size() ; ++j) { 

table.set(j, A.get(i).get(j) ? table.get(j) + 1 : ®); 

} 

maxRectArea = Math.max(maxRectArea, calculateLargestRectangle(table)); 

} 

return maxRectArea; 


The largest square 2D subarray containing only Is can be computed similarly, with 
a minor variant on the algorithm in Solution 18.8 on Page 347. 


25.34 Implement Huffman coding 

One way to compress text is by building a code book which maps each character to a 
bit string, referred to as its code word. Compression consists of concatenating the bit 
strings for each character to form a bit string for the entire text. (The codebook, i.e., 
the mapping from characters to corresponding bit strings is stored separately, e.g., in 
a preamble.) 

When decompressing the string, we read bits until we find a string that is in 
the code book and then repeat this process until the entire text is decoded. For the 
compression to be reversible, it is sufficient that the code words have the property 
that no code word is a prefix of another. For example. Oil is a prefix of 0110 but not 
a prefix of 1100. 

Since our objective is to compress the text, we would like to assign the shorter 
bit strings to more common characters and the longer bit strings to less common 
characters. We will restrict our attention to individual characters. (We may achieve 
better compression if we examine common sequences of characters, but this increases 
the time complexity.) 

The intuitive notion of commonness is formalized by the frequency of a character 
which is a number between zero and one. The sum of the frequencies of all the 
characters is 1. The average code length is defined to be the sum of the product of 
the length of each character's code word with that character's frequency. Table 25.1 
shows the frequencies of letters of the English alphabet. 


Table 25.1: English characters and their frequencies, expressed as percentages, in everyday docu¬ 
ments. 


Character 

a 

b 

c 

d 

e 

f 

8 

h 

i 


Frequency 

8.17 

1.49 

2.78 

4.25 

12.70 

2.23 

2.02 

6.09 

6.97 


Character 

j 

k 

1 

m 

n 

o 

P 

q 

r 


Frequency 

0.15 

0.77 

4.03 

2.41 

6.75 

7.51 

1.93 

0.10 

5.99 


Character 

s 

t 

u 

V 

w 

X 

y 

z 


Frequency 

6.33 

9.06 

2.76 

0.98 

2.36 

0.15 

1.97 

0.07 


497 



Given a set of characters with corresponding frequencies, find a code book that has 
the smallest average code length. 

Hint: Reduce the problem from n characters to one on n - 1 characters. 

Solution: The trivial solution is to use fixed length bit strings for each character. To 
be precise, if there are n distinct characters, we can use fig ri] bits per character. If 
all characters are equally likely, this is optimum, but when there is large variation in 
frequencies, we can do much better. 

A natural heuristic is to split the set of characters into two subsets, which have 
approximately equal aggregate frequencies, solve the problem for each subset, and 
then add a 0 to the codes from the first set and a 1 to the codes from the second 
set to differentiate the codes from the two subsets. This approach does not always 
result in the optimum coding, e.g., when the characters are A, B, C, D with frequencies 
{0.4,0.35,0.2,0.05}, it forms the code words 00,10,11,01, whereas the optimum coding 
is 0,10,110, 111. It also requires being able to partition the characters into two subsets 
whose aggregate frequencies are close, which is computationally challenging. 

Another strategy is to assign 0 to the character with highest frequency, solve the 
same problem on the remaining characters, and then prefix those codes with a 1. This 
approach fares very poorly when characters have the same frequency, e.g., if A, B, C, D 
all have frequency 0.25, the resulting coding is 0,10,100, 111, whereas 00,01,10,11 is 
the optimum coding. Intuitively, this strategy fails because it does not take into 
account the relative frequencies. 

Huffman coding yields an optimum solution to this problem. (There may be other 
optimum codes as well.) It is based on the idea that you should focus on the least 
frequent characters, rather than the most frequent ones. Specifically, combine the two 
least frequent characters into a new character, recursively solve the problem on the 
resulting n - 1 characters; then create codes for the two combined characters from the 
code for their combined character by adding a 0 for one and a 1 for the other. 

More precisely, Huffman coding proceeds in three steps: 

(1.) Sort characters in increasing order of frequencies and create a binary tree node 
for each character. Denote the set just created by S. 

(2.) Create a new node u whose children are the two nodes with smallest frequencies 
and assign u's frequency to be the sum of the frequencies of its children. 

(3.) Remove the children from S and add u to S. Repeat from Step (2.) till S consists 
of a single node, which is the root. 

Mark all the left edges with 0 and the right edges with 1. The path from the root 
to a leaf node yields the bit string encoding the corresponding character. 

Applying this algorithm to the frequencies for English characters presented in 
Table 25.1 on the preceding page yields the Huffman tree in Figure 25.8 on the next 
page. The path from root to leaf yields that character's Huffman code, which is listed 
in Table 25.2 on the facing page. For example, the codes for t,e, and z are 000,100, 
and 001001000, respectively. 

The codebook is explicitly given in Table 25.2 on the next page. The average code 
length for this coding is 4.205. In contrast, the trivial coding takes [log 26] = 5 bits for 
each character. 
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Figure 25.8: A Huffman tree for the English characters, assuming the frequencies given in Table 25.1 
on Page 497. 


Table 25.2: Huffman codes for English characters, assuming the frequencies given in Table 25.1 on 
Page 497. 


Character 

Huffman code 

Character 

Huffman code 

Character 

Huffman code 

a 

1110 

j 

001001011 

s 

0111 

b 

110000 

k 

0010011 

t 

000 

c 

01001 

1 

11110 

u 

01000 

d 

mu 

m 

00111 

V 

001000 

e 

100 

n 

1010 

w 

00110 

f 

00101 

o 

1101 

X 

001001010 

8 

110011 

P 

110001 

y 

110010 

h 

0110 

q 

001001001 

z 

001001000 

i 

1011 

r 

0101 




In the implementation below, we use a min-heap of candidate nodes to represent 

S. 


public static class CharWithFrequency { 
public char c; 
public double freq; 
public String code; 

} 

public static class BinaryTree implements Comparable<BinaryTree> { 
public double aggregateFreq; 
public CharWithFrequency s; 
public BinaryTree left, right; 

public BinaryTree (double aggregateFreq, CharWithFrequency s, 
BinaryTree left, BinaryTree right) { 
this .aggregateFreq = aggregateFreq; 
this.s = s; 
this. left = left; 
this. right = right; 
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} 


©Override 

public int compareTo(BinaryTree o) { 

return Double.compare(aggregateFreq, o.aggregateFreq); 

} 


©Override 

public boolean equals(Object obj) { 

if (obj == null || !(obj instanceof BinaryTree)) { 

return false; 

} 

return this == obj ? true 

: aggregateFreq == ((BinaryTree)obj).aggregateFreq; 


} 


©Override 

public int hashCode() { return Objects.hash(aggregateFreq); } 

} 

public static MapcCharacter, String> huffmanEncoding( 

List<CharWithFrequency> symbols) { 

PriorityQueue<BinaryTree> candidates = new PriorityQueue<>(); 

// Add leaves for symbols. 

for (CharWithFrequency s : symbols) { 

candidates.add(new BinaryTree(s.freq, s, null, null)); 

} 


// Keeps combining two nodes until there is one node left, which is the 
// root. 

while (candidates.size () > 1) { 

BinaryTree left = candidates.remove(); 

BinaryTree right = candidates.remove(); 

candidates.add(new BinaryTree(left.aggregateFreq + right.aggregateFreq, 

null, left, right)); 


} 


Map<Character, String> huffmanEncoding = new HashMap<>(); 

// Traverses the binary tree, assigning codes to nodes. 

assignHuffmanCode(candidates.peek(), new StringBuilder(), huffmanEncoding); 
return huffmanEncoding; 


private static void assignHuffmanCode( 

BinaryTree tree, StringBuilder code, 

MapcCharacter, String> huffmanEncoding) { 
if (tree != null) { 
if (tree.s != null) { 

// This node is a leaf. 

huffmanEncoding.put(tree.s.c, code.toString()); 

} else { // Non-leaf node. 
code.append(’©’); 

assignHuffmanCode(tree.left, code, huffmanEncoding); 
code.setLength(code.length() - 1); 
code.append(’ 1 ’ ) ; 

assignHuffmanCode(tree . right , code, huffmanEncoding); 
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Figure 25.9: The area of the horizontal lines is the maximum amount of water that can be trapped by 
the solid region. For this container, it is 1 + 2 + 1 + 3 = 7. 


code.setLength(code.length() - 1); 

} 

} 

} 


Since each invocation of Steps (2.) on Page 498 and (3.) on Page 498 requires two 
extract-min and one insert operation, it takes 0(n log n) time to build the Huffman tree, 
where n is the number of characters. It's possible for the tree to be very skewed. In 
such a situation, the codewords are of length 1,2,3,..., n, so the time to generate the 
codebook becomes 0(1 + 2 + ••• + «) = 0(n 2 ). 

It is exceedingly unlikely that you would be asked for a rigorous proof of optimality 
in an interview setting. The proof that the Huffman algorithm yields the minimum 
average code length uses induction on the number of characters. The induction step 
itself makes use of proof by contradiction, with the two leaves in the Huffman tree 
corresponding to the rarest characters playing a central role. 

25.35 Trapping water 

The goal of this problem is to compute the capacity of a type of one-dimensional 
container. The computation is illustrated in Figure 25.9. 

A one-dimensional container is specified by an array of n nonnegative integers, spec¬ 
ifying the height of each unit-width rectangle. Design an algorithm for computing 
the capacity of the container. 

Hint: Draw pictures, and focus on the extremes. 

Solution: We can get a great deal of insight by visualizing pouring water into the 
container. When the maximum capacity is achieved, the cross-section consists of a 
region in which the water level is nondecreasing, followed by a region in which the 
water level is nonincreasing. The transition from nondecreasing to nonincreasing 
takes place around a maximum entry in A. Let A[m] be a maximum value entry. 
Then we compute the capacity of A[0 : m - 1] and A[m : n - 1] independently. These 
capacities are determined via an iteration. For each entry in A[0 : m - 1] we compute 
the difference between its value entry and the running maximum, and add that to 
the total capacity. We handle A[m : n - 1] analogously. The time complexity is 0(n) 
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to find a maximum, and then 0(n) for each of the two iterations, i.e., the total time 
complexity is 0(n). The space complexity is (9(1)—all that is needed is to record 
several variables. 


public static int calculateTrappingWater(List<Integer> A) { 
if (A.isEmpty()) { 
return ®; 

} 

// Finds the index with maximum height. 
int maxH = getlndexOfMaxElement(A); 

// Calculates the water within A.subList(l, maxH). 
int sum = ®, left = A.get(®); 
for (int i = 1; i < maxH; ++i) { 
if (A.get(i) >= left) { 
left = A.get(i); 

} else { 

sum += left - A.get(i); 

} 

} 

// Calculates the water within A.subList(maxH + 1, A.size() - 1). 
int right = A.get(A.size () - 1); 
for (int i = A.sizeC) - 2; i > maxH; --i) { 
if (A.get(i) >= right) { 
right = A.get(i) ; 

} else { 

sum += right - A.get(i); 

} 

} 

return sum; 


Variant: Solve the water filling problem with an algorithm that accesses A's elements 
in order and can read an element only once. Use minimum additional space. 


25.36 Search for a pair-sum in an abs-sorted array ©• 

An abs-sorted array is an array of numbers in which \A[i]\ < \A[j]\ whenever i < j. 
For example, the array in Figure 25.10, though not sorted in the standard sense, is 
abs-sorted. 


-49 

75 

103 

-147 

164 

-197 

-238 

314 

348 

-422 

A[ 0] 

A[l] 

M 2] 

A[ 3] 

^[4] 

A[ 5] 

A[ 6] 

au\ 

A[ 8] 

A[ 9] 


Figure 25.10: An abs-sorted array. 


Design an algorithm that takes an abs-sorted array A and a number K, and returns a 
pair of indices of elements in A that sum up to K. For example, if the input to your 
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algorithm is the array in Figure 25.10 on the facing page and K = 167, your algorithm 
should output (3,7). Output (-1,-1) if there is no such pair. 

Hint: This problem easy to solve with 0(n) additional space—why? To solve it with 0(1) 
additional space, first assume all elements are positive. 

Solution: First consider the case where the array is sorted in the conventional sense. 
In this case we can start with the pair consisting of the first element and the last 
element: (A[0] f A[n - 1]). Let s = A[0] + A[n - 1]. If s = K, we are done. If s < K, 
we increase the sum by moving to pair (A[l\,A[n - 1]). We need never consider A[0]; 
since the array is sorted, for all i, A[0] + A[i\ < A[0] + A[n-1] = s < K. If s > K, we can 
decrease the sum by considering the pair (A[0],A[n - 2]); by analogous reasoning, we 
need never consider A[n - 1] again. We iteratively continue this process till we have 
found a pair that sums up to K or the indices meet, in which case the search ends. 
This solution works in 0(n) time and 0(1) space in addition to the space needed to 
store A. 

This approach will not work when the array entries are sorted by absolute value. 
In this instance, we need to consider three cases: 

(1.) Both the numbers in the pair are negative. 

(2.) Both the numbers in the pair are positive. 

(3.) One is negative and the other is positive. 

For Cases (1.) and (2.), we can run the above algorithm separately by just limiting 
ourselves to either positive or negative numbers. For Case (3.), we can use the same 
approach where we have one index for positive numbers, one index for negative 
numbers, and they both start from the highest possible index and then go down. 

private static class IndexPair { 
public Integer indexl; 
public Integer index2; 

public IndexPair(Integer indexl, Integer index2) { 
this.indexl = indexl; 
this.index2 = index2; 

} 

} 

private static interface BooleanCompare { 

public boolean compare(Integer indexl, Integer index2); 

} 

private static class CompareLess implements BooleanCompare { 

©Override 

public boolean compare(Integer ol, Integer o2) { return ol < o2; } 
public static final CompareLess LESS = new CompareLess(); 

} 

private static class CompareGreaterEqual implements BooleanCompare { 

©Override 

public boolean compare(Integer ol, Integer o2) { return ol >= o2; } 
public static final CompareGreaterEqual GREATER_OR_EQUAL 
= new CompareGreaterEqual(); 

} 
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public static IndexPair findPairSumK(List<Integer> A, int k) { 

IndexPair result = findPosNegPair(A, k); 
if (result.indexl == -1 && result.index2 == -1) { 

return k >= ® 

? findPairUsingComp(A, k, CompareLess.LESS) 

: findPairUsingComp(A, k, CompareGreaterEqual.GREATER_OR_EQUAL); 

} 

return result; 

} 

private static IndexPair findPairUsingComp(List<Integer> A, int k, 

BooleanCompare comp) { 

IndexPair result = new IndexPair(®, A.size() - 1); 
while (result.indexl < result.index2 

&& comp.compare(A.get(result.indexl) , ®)) { 
result.indexl = result.indexl + 1; 

} 

while (result.indexl < result.index2 

&& comp.compare(A.get(result.index2), ®)) { 
result.index2 = result.index2 - 1; 

} 


while (result.indexl < result.index2) { 

if (A.get(result.indexl) + A.get(result.index2) == k) { 
return result; 

} else if (comp.compare(A.get(result.indexl) + A.get(result.index2), 
do { 

result.indexl = result.indexl + 1; 

} while (result.indexl < result.index2 

&& comp.compare(A.get(result.indexl), ®)); 

} else { 
do { 

result.index2 = result.index2 - 1; 

} while (result.indexl < result.index2 

&& comp.compare(A.get(result.index2), ®)); 


} 


} 


return new IndexPair(-1, -1); // No answer. 


} 


k)) { 


private static IndexPair findPosNegPair(Listdnteger> A, int k) { 
// result, first for positive, and result.second for negative. 
IndexPair result = new IndexPair(A.size() - 1, A.size() - 1); 

// Find the last positive or zero. 

while (result.indexl >= ® && A.get(result.indexl) < ®) { 
result.indexl = result.indexl - 1; 

} 


// Find the last negative. 

while (result.index2 >= ® && A.get(result.index2) >= ®) { 
result.index2 = result.index2 - 1; 

} 

while (result.indexl >= ® && result.index2 >= ®) { 

if (A.get(result.indexl) + A.get(result.index2) == k) { 
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return result; 

} else if (A.get(result.index 1) + A.get(result.index2) > k) { 
do { 

result.indexl = result.indexl - 1; 

} while (result. indexl >= ® &<& A . get (result. indexl) < ®) ; 

} else { // A.get(result. first) + A.get(result.second) < k. 
do { 

result.index2 = result.index2 - 1; 

} while (result.index2 >= ® && A.get(result.index2) >= ®) ; 

} 

} 

return new IndexPair(-1, -1); // No answer. 


Since the computation entails three passes through the array, each of which takes 0(n) 
time, its time complexity is 0(n). No memory is allocated, so the space complexity is 
0 ( 1 ). 

A much simpler solution is based on a hash table (Chapter 13) to store all the 
numbers and then for each number x in the array, lookup K - x in the hash table. If 
the hash function does a good job of spreading the keys, the time complexity for this 
approach is 0(n). However, it requires 0(n) additional storage. 

If the array is sorted on elements (and not absolute values), for each A [i] we can use 
binary search to find K - A[i]. This approach uses 0(1) additional space and has time 
complexity 0(n log ft). However, it is strictly inferior to the two pointer technique 
described at the beginning of the solution. 

Variant: Design an algorithm that takes as input an array of integers A, and an integer 
K, and returns a pair of indices i and j such that A[j] - A[i] = K, if such a pair exists. 


25.37 The heavy hitter problem © 

This problem is a generalization of Problem 18.5 on Page 341. In practice we may not 
be interested in a majority token but all tokens whose count exceeds say 1% of the 
total token count. It is fairly straightforward to show that it is impossible to compute 
such tokens in a single pass when you have limited memory. However, if you are 
allowed to pass through the sequence twice, it is possible to identify the common 
tokens. 

You are reading a sequence of strings separated by whitespace. You are allowed to 
read the sequence twice. Devise an algorithm that uses 0(k) memory to identify the 
words that occur more than | times, where n is the length of the sequence. 

Hint: Maintain a list of k candidates. 

Solution: This is essentially a generalization of Problem 18.5 on Page 341. Here 
instead of discarding two distinct words, we discard k distinct words at any given 
time and we are guaranteed that all the words that occurred more than \ times the 
length of the sequence prior to discarding continue to appear more than \ times in the 
remaining sequence. To implement this strategy, we need a hash table of the current 
candidates. 
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public static List<String> searchFrequentTtems(Iterable<String> input, 

int k) { 

// Finds the candidates which may occur > n / k times. 

String buf = 

Map<String , Integer> hash = new HashMap<>(); 
int n = 8; // Counts the number of strings. 

Iterator<String> sequence = input.iterator(); 
while (sequence.hasNext()) { 
buf = sequence.next(); 

hash.put(buf, hash.containsKey(buf) ? hash.get(buf) +1 : 1); 

++n; 

// Detecting k items in hash, at least one of them must have exactly one 
// in it. We will discard those k items by one for each. 
if (hash.sizeO == k) { 

List<String> delKeys = new ArrayList<>(); 

for (Map.Entry<String , Integer> entry : hash.entrySet()) { 
if (entry.getValue() - 1 == 8) { 
delKeys . add (entry .getKeyO) ; 

} 

} 

for (String s : delKeys) { 
hash.remove(s); 

} 

for (Map.Entry<String , Integer> e : hash.entrySet()) { 
hash .put (e . getKeyO , e.getValue() - 1); 

} 

} 

} 

// Resets hash for the following counting. 
for (String it : hash.keySet()) { 
hash.put(it, 8) ; 

} 

// Counts the occurrence of each candidate word. 
sequence = input.iterator () ; 
while (sequence.hasNext()) { 
buf = sequence.next(); 

Integer it = hash.get(buf); 
if (it != null) { 

hash.put(buf, it + 1); 

} 

} 

// Selects the word which occurs > n / k times. 

List<String> ret = new ArrayList<>(); 

for (Map.Entry<String, Integer> it : hash.entrySet()) { 
if (n * 1.8 / k < (double)it.getValue()) { 
ret. add (it .getKeyO) ; 

} 

} 

return ret; 
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The code may appear to take 0(nk) time since the inner loop may take k steps 
(decrementing count for all k entries) and the outer loop is called n times. However 
each word in the sequence can be erased only once, so the total time spent erasing is 
0(n) and the rest of the steps inside the outer loop run in 0(1) time. 

The first step yields a set S of not more than k words; set S is a superset of the 
words that occur greater than | times. To get the exact set, we need to make another 
pass over the sequence and count the number of times each word in S actually occurs. 
We return the words in S which occur more than J times. 


25.38 Find the longest subarray whose sum < ©< 

Here we consider finding the longest subarray subject to a constraint on the subarray 
sum. For example, for the array in Figure 25.11, the longest subarray whose subarray 
sum is no more than 184 is A [3 : 6]. 


431 

-15 

639 

342 

-14 

565 

-924 

635 

167 

-70 

A[ 0] 

A[ 1] 

A[ 2] 

A[3] 

A[ 4] 

A[5] 

A[6] 

A[7] 

A[8] 

A[ 9] 


Figure 25.11: An array for the longest subarray whose sum < k problem. 


Design an algorithm that takes as input an array A of n numbers and a key k, and 
returns the length of a longest subarray of A for which the subarray sum is less than 
or equal to k . 

Hint: When can you be sure that an index i cannot be the starting point of a subarray with the 
desired property, without looking past /? 

Solution: The brute-force solution entails computing £*_.A[fc], i.e., the sum of the 
element in A[i : j], for all 0 < i < j < n- 1, where n is the length of A. Let P be the prefix 
sum array for A, i.e., P[i] = Tj[=o A[k\, P can be computed in a single iteration over A 
in 0(n) time. Note that the sum of the elements in the subarray A [i: j] is P[j] - P[i -1] 
(for convenience, take P[-l] = 0). The time complexity of the brute-force solution is 
0(n 2 ), and the additional space complexity is 0(n) (the size of P). 

The trick to improving the time complexity to 0(n) comes from the following 
observation. Suppose u < v and P[u] > P[v]. Then u will never be the ending point 
of a solution. The reason is that for any zu < u, A[w : v] is longer than A[zv : u] and if 
A[w : u] satisfies the sum constraint, so must A[zv : v]. This motivates the definition 
of the array O: set Q[z] = min(P[z], Q[i + 1]) for i < n - 1, and Q[n - 1] = P[n - 1]. 

Let a < b be indices of elements in A. Define M a/b to be the minimum possible sum 
of a subarray beginning at a and extending to b or beyond. Note that Mo /b = Q[b ], and 
M tt/b = Q[b] - P[a - 1], when a > 0. If M a/b > k, no subarray starting at a that includes b 
can satisfy the sum constraint, so we can increment a. If M a>b < k, then we are assured 
there exists a subarray of length b - a +1 satisfying the sum constraint, so we compare 
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the length of the longest subarray satisfying the sum constraint identified so far to 
b - a + 1 and conditionally update it. Consequently, we can increment b. 

Suppose we initialize a and b to 0 and iteratively perform the increments to a and 
b described above until b = n. Then we will discover the length of a maximum length 
subarray that satisfies the sum constraint. We justify this claim after presenting an 
implementation of these ideas below. 

public static int findLongestSubarrayLessEqualK(List<Integer> A, int k) { 

// Build the prefix sum according to A. 

Listdnteger> prefixSum = new ArrayList<>() ; 
int sum = 0; 
for (int a : A) { 
sum += a; 

prefixSum.add(sum); 

} 

// Early returns if the sum of A is smaller than or equal to k. 
if (prefixSum.get(prefixSum.size() - 1) <= k) { 
return A.size () ; 

} 


// Builds minPrefixSum. 

List dnteger > minPrefixSum = new ArrayList o(prefixSum) ; 
for (int i = minPrefixSum.size() - 2; i >= ®; --i) { 
minPrefixSum.set(i, 

Math.min(minPrefixSum.get(i), minPrefixSum.get(i 


} 


l))); 


int a = ®, b = ®, maxLength = ®; 
while (a < A.sizeO && b < A.sizeO) { 

int minCurrSum = a > ® ? minPrefixSum.get(b) - prefixSum.get(a - 1) 
: minPrefixSum.get(b); 

if (minCurrSum <= k) { 

int currLength = b - a + 1; 
if (currLength > maxLength) { 
maxLength = currLength; 

} 

++b; 

} else { // minCurrSum > k. 

++3. I 

} 

} 

return maxLength; 


Now we argue the correctness of the program. Let A[a* : b*] be a maximum length 
subarray that satisfies the sum constraint. Note that we increment b until M a/b > k. In 
particular, when we increment a to a + 1, A [a : b - 1] does satisfy the sum constraint, 
but A[a : b] does not. This implies A[a : b - 1] is the longest subarray starting at a that 
satisfies the sum constraint. 

The iteration ends when b = n. At this point, we claim a > a*. If not, then A [a : n — 1] 
satisfies the sum constraint, since we incremented b to n, and (n-l)-a + l > b* - a* +1, 
contradicting the optimality otA[a* : b*]. Therefore, a must be assigned to a* at some 
iteration. At this point, b < b* since A[a* - 1 : b - 1] satisfies the sum constraint. For, 
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if b > b*, then (b - 1) - (a* - 1) + 1 = b — a* + 1 > b* - a* + 1, violating the maximality 
of A[a* : fr*]. Since b < b* and a = a*, the algorithm will increment b till it becomes b * 
(since A [a* : b*] satisfies the sum constraint), and thus will identify b* - a* + 1 as the 
optimum solution. 

Variant: Design an algorithm for finding the longest subarray of a given array such 
that the average of the subarray elements is < k. 


25.39 Road network © 

The California Department of Transportation is considering adding a new section 
of highway to the California Highway System. Each highway section connects two 
cities. City officials have submitted proposals for the new highway—each proposal 
includes the pair of cities being connected and the length of the section. 

Write a program which takes the existing highway network (specified as a set of 
highway sections between pairs of cities) and proposals for new highway sections, 
and returns the proposed highway section which leads to the most improvement in 
the total driving distance. The total driving distance is defined to be the sum of the 
shortest path distances between all pairs of cities. All sections, existing and proposed, 
allow for bi-directional traffic, and the original network is connected. 

Hint: Suppose we add a new section from b s tobf. If the shortest path from u to v passes through 
this section, what must be true of the part of the path from u to b s ? 

Solution: Note that we cannot add more than one proposal to the existing network 
and run a shortest path algorithm—we may end up with a shortest path which uses 
multiple proposals. 

The brute-force approach would be to first compute the shortest path distances 
for all pairs in the original network. Then consider the new sections, one-at-a-time, 
and then compute the new shortest path distances for all pairs, recording the total 
improvement. The all-pairs shortest path problem can be solved in time 0(n 3 ) using 
the Floyd-Warshall algorithm, leading to an overall 0(kn 3 ) time complexity. 

We can improve upon this by running the all pairs shortest paths algorithm just 
once. Let S(u, v) be the 2D array of shortest path distances for each pair of cities. Each 
proposal p is a pair of cities {x, y). For the pair of cities (a, b), the best we can do by 
using proposal p is min(S(a, b), S{a, x) + d(x, y) + S(y, b), S{a, y) + d(y, x) + S(x, b)) where 
d(x, y) is the distance of the proposed highway p between x and y. This computation 
is (9(1) time, so we can evaluate all the proposals in time proportional to the number 
of proposals times the number of pairs after we have computed the shortest path 
between each pair of cities. This results in an 0(n 3 + kn 2 ) time complexity, which 
improves substantially on the brute-force approach. 

public static class HighwaySection { 
public int x, y; 
public double distance; 

public HighwaySect ion(int x, int y, double distance) { 
this.x = x; 
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this.y = y; 

this .distance = distance; 

} 

} 

public static HighwaySection findBestProposals(List<HighwaySection> H, 

List<HighwaySection> P, 
int n) { 

// G stores the shortest path distances between all pairs of vertices. 

List<List<Double>> G = new ArrayList<>(n); 
for (int i = ®; i < n; ++i) { 

G.add(new ArrayList(Collections.nCopies(n, Double.MAX_VALUE))); 

} 

for (int i = ®; i < n; ++i) { 

G.get(i) . set(i , ©.®); 

} 

// Builds an undirected graph G based on existing highway sections H. 
for (HighwaySection h : H) { 

G.get(h.x).set(h.y, h.distance); 

G.get(h.y).set(h.x, h.distance); 

} 

// Performs Floyd Warshall to build the shortest path between vertices. 
floydWarshall(G); 

// Examines each proposal for shorter distance for all pairs. 
double bestDistanceSaving = Double.MIN_VALUE; 

HighwaySection bestProposal = new HighwaySection(-l, -1, ®.®); // Default. 
for (HighwaySection p : P) { 
double proposalSaving = ®.®; 
for (int a = ®; a < n; ++a) { 
for (int b = ®; b < n; ++b) { 

double saving = G.get(a).get(b) - (G.get(a).get(p.x) + p.distance 

+ G.get(p.y).get(b)); 

proposalSaving += saving > ®.® ? saving : ©.©; 

} 

} 

if (proposalSaving > bestDistanceSaving) { 
bestDistanceSaving = proposalSaving; 
bestProposal = p; 

} 

} 

return bestProposal; 


private static void floydWarshall(List<List<Double>> G) { 
for (int k = ®; k < G.sizeO; ++k) { 
for (int i = ®; i < G.sizeO; ++i) { 
for (int j = ®; j < G.sizeO; ++j) { 

if (G.get(i).get(k) != Double.MAX.VALUE 

&& G.get(k).get(j) != Double.MAX.VALUE 

&& G. get (i) . get (j ) > G. get (i) . get (k) + G . get (k) . get ( j ) ) { 
G. get (i) . set (j , G. get (i) . get (k) + G . get (k) . get ( j ) ) ; 

} 

} 

} 
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} 


} 


25.40 Test if arbitrage is possible 

You are exploring the remote valleys of Papua New Guinea, one of the last uncharted 
places in the world. You come across a tribe that does not have money—instead it 
relies on the barter system. A total of n commodities are traded and the exchange 
rates are specified by a 2D array. For example, three sheep can be exchanged for 
seven goats and four goats can be exchanged for 200 pounds of wheat. 

Transaction costs are zero, exchange rates do not fluctuate, fractional quantities of 
items can be sold, and the exchange rate between each pair of commodities is finite. 
Table 4.4 on Page 35 shows exchange rates for currency trades, which is similar in 
spirit to the current problem. 

Design an efficient algorithm to determine whether there exists an arbitrage—a way 
to start with a single unit of some commodity C and convert it back to more than one 
unit of C through a sequence of exchanges. 

Hint: The effect of a sequence of conversions is multiplicative. Can you recast the problem so 
that it can be calculated additively? 

Solution: We define a weighted directed graph G = (V, E = V x V), where V corre¬ 
sponds to the set of commodities. The weight w(e) of edge e = ( u , v) is the amount of 
commodity v we can buy with one unit of commodity u. Observe that an arbitrage 
exists if and only if there exists a cycle in G whose edge weights multiply out to more 
than 1. 

Create a new graph G' = ( V,E ) with weight function w'(e) = -lgip(e). Since 
1 g(fl X b) = lg a + lg b, there exists a cycle in G whose edge weights multiply out to 
more than 1 if and only if there exists a cycle in G' whose edge weights sum up to 
less than lg 1 = 0. (This property is true for logarithms to any base, so if it is more 
efficient for example to use base-e, we can do so.) 

The Bellman-Ford algorithm detects negative-weight cycles. Usually, finding a 
negative-weight cycle is done by adding a dummy vertex s with 0-weight edges to 
each vertex in the given graph and running the Bellman-Ford single-source shortest 
path algorithm from s. However, for the arbitrage problem, the graph is complete. 
Hence, we can run Bellman-Ford algorithm from any single vertex, and get the right 
result. 

public static boolean isArbitrageExist(List<List<Double>> G) { 

// Transforms each edge in G. 
for (List<Double> edgeList : G) { 

for (int i = Q; i < edgeList.size(); i++) { 

edgeList.set(i, -Math.log 1®(edgeList.get(i))); 

} 

} 

// Uses Bellman-Ford to find negative weight cycle. 
return bellmanFord(G, Q); 
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} 


private static boolean bellmanFord(List<List<Double>> G, int source) { 
List<Double> disToSource 

= new ArrayList<>(Collections.nCopies(G.size(), Double.MAX_VALUE)); 
disToSource.set(source, ®.®); 

for (int times = 1; times < G.sizeO; ++times) { 
boolean haveUpdate = false; 
for (int i = ®; i < G.sizeO; ++i) { 

for (int j = ®; j < G.get(i).size() ; ++j) { 
if (disToSource.get(i) != Double.MAX_VALUE 

&& disToSource.get(j) > disToSource.get(i) + G.get(i).get(j)) { 
haveUpdate = true; 

disToSource.set (j , disToSource.get(i) + G.get(i).get(j)) ; 

} 

} 

} 

// No update in this iteration means no negative cycle. 
if (!haveUpdate) { 

return false; 

} 

} 

// Detects cycle if there is any further update. 
for (int i = ®; i < G.sizeO; ++i) { 

for (int j = ®; j < G.get(i).size() ; ++j) { 
if (disToSource.get(i) != Double.MAX.VALUE 

<&& disToSource.get(i) > disToSource.get(i) + G.get(i).get(j)) { 
return true; 

> 

} 

} 

return false; 


The time complexity of the general Bellman-Ford algorithm is 0(|V1|E|). Here, \E\ = 
0(\V\ 2 ) and \ V\ = n, so the time complexity is 0(n 3 ). 
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Part V 

Notation, and Index 



Notation 


To speak about notation as the only way that you can guarantee 
structure of course is already very suspect. 

— E. S. Parker 


We use the following convention for symbols, unless the surrounding text specifies 
otherwise: 

A ^-dimensional array 

L linked list or doubly linked list 

S set 
T tree 
G graph 

V set of vertices of a graph 
E set of edges of a graph 


Symbolism 

Meaning 

(dk-i ••- do) r 

radix-r representation of a number, e.g., (1011) 2 

[o Sb x 

logarithm of x to the base b 

Igx 

logarithm of x to the base 2 

|S| 

cardinality of set S 

S\T 

set difference, i.e., S n T, sometimes written as S - T 

M 

absolute value of x 

L*J 

greatest integer less than or equal to x 

M 

smallest integer greater than or equal to x 

(a 0 ,ai,... ,a„-i) 

sequence of n elements 

L m f(k) 

sum of all f(k) such that relation R(k) is true 

min R(jt , f(k) 

minimum of all f(k) such that relation R(k) is true 

max R( *) f(k) 

maximum of all f(k) such that relation R(k) is true 

LLm 

shorthand for f(k) 

[a | R(a )| 

set of all a such that the relation R(a) = true 

U,r] 

closed interval: [x \ l < x < r] 

U,r) 

left-closed, right-open interval: {x \ l < x < r) 

\a,b,...) 

well-defined collection of elements, i.e., a set 

A, or A[i] 

the zth element of one-dimensional array A 

A[i : ;] 

subarray of one-dimensional array A consisting of ele¬ 
ments at indices i to j inclusive 

y4[!'][;'] or A[i, j] 

the element in ith row and yth column of 2D array A 
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M.h • h][ji : ji\ 


0 

n\ 

0(f(n)) 
x mod y 
x®y 
x*y 

null 

0 

oo 

x^y 

*» y 


2D subarray of 2D array A consisting of elements from 
z'lth to f 2 th rows and from /ith to ; 2 tb column, inclusive 
binomial coefficient: number of ways of choosing k ele¬ 
ments from a set of n items 

n-factorial, the product of the integers from 1 to n, inclu¬ 
sive 

big-oh complexity of f(n), asymptotic upper bound 
mod function 
bitwise-XOR function 
x is approximately equal to y 

pointer value reserved for indicating that the pointer does 
not refer to a valid address 
empty set 

infinity: Informally, a number larger than any number, 
much less than 
much greater than 
logical implication 
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Index of Terms 


2D array, 85, 87-90, 92, 197, 296, 312, 313, 315, 
316, 354, 357, 360, 396, 494, 509, 511, 
514, 515 

2D subarray, 85,494-497, 515 
0(1) space, 3, 13,15, 26, 27, 40, 65, 69, 70, 79, 89, 
91,100, 120, 122,124-128,135, 152, 
158,165,204,206,216,234,239,253, 
259,305, 329, 341,345,438,440,441, 
444,446,449,451,463,464,490,491, 
502,503,505 

0-1 knapsack problem, 317 

acquired immune deficiency syndrome, see AIDS 
adjacency list, 352, 352, 397 
adjacency matrix, 352, 352 
AIDS, 405 

algorithm patterns, 30 
all pairs shortest paths, 368, 509 
alternating sequence, 331 
amortized, 131,214 
amortized analysis, 207 
API, 28, 28,145,173,186, 395,400,404, 469 
application programming interface, see API 
approximation algorithm, 41 
arbitrage, 35, 35, 405, 511 
array, 1-4,13,15, 25, 25, 26-28, 34, 36-38, 40, 46, 
60, 62, 74, 76, 78, 82, 100, 127, 131, 
142,145,179,181,186-188,190-193, 
198,200, 203,207,214, 234, 236,237, 
239,254,265,267,271,305,315,330- 
332, 340, 347, 357, 390, 438-441,443, 
460,461,463,470,481, 483, 490, 502, 
503, 505, 507 
bit, see bit array 
ascending sequence, 332 
AVL tree, 29 

backtracking, 296 
Bellman-Ford algorithm, 35, 511 

for negative-weight cycle detection, 511 
BFS, 4, 300, 353, 353, 354, 355, 358-361, 364-367 


BFS tree, 355, 365 

binary search, 3,4,15,21,40,41, 85,187,188,190, 
192,193,196,234,237,259, 282, 339, 
399,461,505 

binary search tree, 3,24,25,28, see BST, 207 
AVL tree, 29 
deletion from, 25,29 

height of, 254,259,262,271,274,275,473 
red-black tree, 29,254 

binary tree, see also binary search tree, 25, 28, 29, 
143-145,150-152,154-161,163,165- 
169,171-173,175,216,254,256,261, 
262,295, 352,454,498 
complete, 151,175 
full, 151 

height of, 25,28,29,151-155,161-164,455 

left-skewed, 151 

perfect, 151,171 

right-skewed, 151 

skewed, 151 

binomial coefficient, 314, 315, 515 
bipartite graph, 368 

bit array, 21, 86, 87, 203, 290, 393, 394, 445 
bitonic sequence, 332,332 
Boyer-Moore algorithm, 34 
breadth-first search, see BFS 
BST, 14, 29, 29, 30, 38,185, 208, 234, 254, 256-261, 
263, 271,272,274-278,470,472,473, 
476 

busy wait, 379 
caching, 390 

Cascading Style Sheet, see CSS 

case analysis, 30, 32, 33, 41, 88,121, 244, 299,436 

central processing unit, see CPU 

CGI, 407 

chessboard, 37,232,233, 285, 287, 445 
mutilated, 36 

child, 151,168, 272, 275, 352,468,498 

circular queue, see also queue 

closed interval, 29, 242, 337,475,478, 514 
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CNF-SAT, 41, 41 
code 

Huffman, 497-499, 501 
coin changing, 333 
Collatz conjecture, 230, 230, 386 
coloring, 357 
column constraint, 86, 87 
combination, 38, 306 
Commons Gateway Interface, see CGI 
complete binary tree, 151, 151, 152,175 
height of, 151 
complexity analysis, 40 
concrete example, 30, 30, 32 
concurrency, 5, 22,173 

conjunctive normal form satisfiability, see CNF- 
SAT 

connected component, 351, 351, 364 
connected directed graph, 351 
connected graph, 30 
connected undirected graph, 351 
connected vertices, 351, 351 
constraint, 1, 27,142, 257,285, 320, 334, 368, 369, 
383, 385,399,488,492,507 
column, 86,87 
hard, 396 
placement, 249 
row, 86, 87 
soft, 396 
stacking, 285 
sub-grid, 86 
synchronization, 377 
convex sequence, 331 
counting sort, 234,246 
CPU, 32, 390, 407 
CSS, 155, 407 

DAG, 350, 350, 369 

data center, 406 

data structure, 24 

data structures, 24 

database, 335, 390, 394,403,404 

deadlock, 375 

decision tree, 404 

decomposition, 389, 390 

deletion 

from binary search trees, 25, 29 

from doubly linked lists, 142 

from hash tables, 25,207 

from heap, 28 

from heaps, 25 

from linked list, 25 

from max-heaps, 175 

from queues, 146 

from singly linked lists, 122 

from stacks, 146 


depth 

of a node in a binary search tree, 258 
of a node in a binary tree, 25,151, 151, 152 
of the function call stack, 40 
depth-first search, 40, see DFS 
deque, 142, 148 

dequeue, 4,142,144-149,453, 454 
DFS, 353, 353, 354, 355, 357-359, 362, 363 
diameter 

of a tree, 300 

Dijkstra's algorithm, 4, 370, 372 
directed acyclic graph, see DAG, 351 
directed graph, 350, see also directed acyclic 
graph, see also graph, 350, 351, 359, 
363, 368 

connected directed graph, 351 
weakly connected graph, 351 
weighted, 511 
discovery time, 354 
distance 

Levenshtein, 309,310,312,391 
distributed memory, 374 
distribution 

of the numbers, 390 

divide-and-conquer, 2, 3, 14, 31, 34, 36, 37, 282, 
300, 303, 305, 368,486 

divisor, 72 

greatest common divisor, 435 
DNS, 407 

Document Object Model, see DOM 
DOM, 155,407 

Domain Name Server, see DNS 
double-ended queue, see deque 
doubly linked list, 25, see also linked list, 27,112, 
112,116,142,220,472, 514 
deletion from, 142 

DP, 14,38, 38, 41, 303, 305, 307, 325, 333,493 
dynamic programming, 3, see DP, 38, 303 

edge, 35, 350, 352, 354, 362-366, 368-370, 511 
capacity of, 368 
weight of, 35,370, 511 
edge set, 352, 368 
elimination, 40,187 

enqueue, 142,144,146,147,149,258,400,453,454 
Ethernet, 300 

Extensible Markup Language, see XML 
extract-max, 175,184,185 
extract-min, 178, 400, 501 

Fibonacci heap, 24 
Fibonacci number, 303 
finishing time, 354 

first-in, first-out, 28, see also queue, 142,146 
fractional knapsack problem, 320 
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free tree, 352, 352 

full binary tree, 151,151 

function 

hash, see hash function 
recursive, 36, 272,296 

garbage collection, 374 
lazy, 214 

GCD, 435,436,443,444 
generalization principle, 37 
graph, 35, 40, 350, see also directed graph, 350, 
351,352, see also tree 
bipartite, 368 

graph modeling, 30, 35, 354 
graphical user interfaces, see GUI 
greatest common divisor, see GCD 
greedy, 31, 38, 38, 39, 333 
greedy algorithm, 5, 21 
GUI, 374, 395 

hard constraint, 396 

hash code, 110, 207, 207, 208, 232, 233, 392, 393, 
395,398,400,465 

hash function, 14, 25, 29, 29, 110, 207, 208, 231- 
233,392-394,400,465,505 
hash table, 4, 21, 24, 25, 28, 29, 38, 82, 94, 118, 
153,166,207-209,212,214,215,220, 
224,230,231,363, 386,391-393,400, 
463-465,468, 505 
deletion from, 25,207 
lookup of, 24,25,29,207,215,224,391,464, 
505 

head 

of a deque, 142,147-149 
of a linked list, 112,117,118,123,471 
of a queue, 148,215,453 
heap, 24, 25, 28, 175, 180,183-186,235, 370,401 
deletion from, 28 
Fibonacci, 24 
insertion of, 184 
max-heap, 175, 235 
min-heap, 175,235 
heapsort, 234 
height 

of a binary search tree, 254, 259, 262, 271, 
274,275,473 

of a binary tree, 25, 28, 29, 151, 151, 152- 
155,161-164, 455 
of a building, 141, 347,485,486 
of a complete binary tree, 151 
of a event rectangle, 240 
of a line segment, 29,30,475,478 
of a perfect binary tree, 151 
of a player, 249 
of a stack, 154 


of a tree, 161 

height-balanced BST, 470 
highway network, 509 
HTML, 393, 402, 403, 404, 405, 407 
HTTP, 341, 381, 403,404,407 
Huffman code, 497-499, 501 
Huffman tree, 498,499 
HyperText Markup Language, see HTML 
Hypertext Transfer Protocol, see HTTP 

I/O, 21, 177, 381 
IDE, 13, 15 
in-place sort, 234 
input/output, see I/O 

integral development environment, see IDE 

International Standard Book Number, see ISBN 

Internet Protocol, see IP 

intractability, 41,41 

invariant, 31, 39,339,340 

inverted index, 236, 396, 397 

IP, 106,106,202,203,341,407 

ISBN, 214,214,215 

iterative refinement, 30, 33 

JavaScript Object Notation, see JSON 
JSON, 402,404,407 

knapsack problem 
0-1, 317 
fractional, 320 

LAN, 300 

last-in, first-out, 28, see also stack, 131,140,146 
lazy garbage collection, 214 
LCA, 155,155,156,157,216,261 
leaf, 25,151,159,160,168,169,171,216,498 
Least Recently Used, see LRU 
least significant bit, see LSB 
left child, 150, 151, 153, 161, 167, 257, 278, 279, 
352,455 

left subtree, 150-152, 154,160, 166, 254, 256-258, 
262, 263 

left-closed, right-open interval, 514 
length 

of a sequence, 182, 462 
of a string, 393 

level 

of a tree, 151 

Levenshtein distance, 309, 309, 310, 312, 391 
line segment, 29,30,475,476,478 
height of, 29, 30, 475,478 
linked list, 25,28,112, 215, 396, 514 
list, 25, see also singly linked list, 117, 118, 120, 
121,123,131,142,168, 207,234,470 
postings, 139, 449,450 
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livelock, 375 
load 

of a hash table, 207 
local area network, see LAN 
lock 

deadlock, 375 
livelock, 375 

longest alternating subsequence, 331 

longest bitonic subsequence, 332 

longest convex subsequence, 331 

longest nondecreasing subsequence, 330,330,331 

longest path, 300, 301, 369 

longest weakly alternating subsequence, 331 

lowest common ancestor, see LCA, 261 

LRU, 214 

LSB, 48, 50 

matching, 368 

maximum weighted, 368 
of strings, 25, 36, 94,330 
matrix, 91, 312, 352, 397 
adjacency, 352 
Boolean, 35 

multiplication of, 374,398 
matrix multiplication, 374,398 
max-heap, 175,181,183,184,235 
deletion from, 25,175 
maximum flow, 368, 368 
maximum weighted matching, 368 
median, 33,182, 214 
merge sort, 178,234,282 

min-heap, 25,28,175,177,178,180,183,234,235, 
385, 386,390,400,457,458,462 
in Huffman's algorithm, 499 
minimum spanning tree, 368 
most significant bit, see MSB 
MSB, 48,158, 203 
MST, 368 

multicore, 374, 386 
mutex, 173,383 
mutilated chessboard, 36 

negative-weight cycle, 511 
network, 9, 302, 374,407,408 
highway, 509 
local are network, 300 
network bandwidth, 341, 390 
network layer, 399 
network route, 180 
network session, 79 
network stack, 399 
network traffic control, 452 
social, 393, 398 
network bandwidth, 341, 390 
network layer, 399 


network session, 79 

node, 28,29,139,140,144,150-166,168,169,171- 
173, 208,216,220,254,256-262,264, 
265, 272,275,278,279, 300, 301, 352, 
454-456, 468,470-473, 498 

NP, 41 

NP-complete, 481 
NP-hard, 333 

operating system, see OS 
ordered pair, 465 
ordered tree, 352, 352 
OS, 5, 389,404 
overflow 

integer, 188,231, 314 
overlapping intervals, 241 

palindrome, 127,212,212,311 
parallel algorithm, 41 
parallelism, 374, 375, 390 
parent-child relationship, 151,352 
partition, 231,368,398,400,401,481 
path, 350 

shortest, see shortest paths 
PDF, 10,396 

perfect binary tree, 151,151,152,171 
height of, 151 

permutation, 74-76, 78, 81, 82,443,444 
random, 80, 81 
uniformly random, 81 
placement constraint, 249 
Polish notation, 136 
Portable Document Format, see PDF 
postings list, 139,449,450 
power set, 289,289 
prefix 

of a string, 467-469,497 
prefix sum, 34,507 
primality, see prime 
prime, 40,72,72,232, 254 
production sequence, 366 

queue, 24, 25,28,142,142,144-146,148,186,215, 
258, 355, 358, 385,400, 453,454 
deletion from, 146 
quicksort, 3, 26, 62, 234, 282, 305 

Rabin-Karp algorithm, 109 
race, 375-377 
radix sort, 235 

RAM, 181,196, 202, 390, 396-398,400,407 
random access memory, see RAM 
random number generator, 56, 78, 79, 81, 83, 85 
random permutation, 80, 81 
uniformly, 81 
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randomization, 207 
reachable, 350, 353 

recursion, 14, 31, 36, 36, 37, 38,139,164, 315,436, 
479,488 

recursive function, 36, 272,296 
red-black tree, 29, 254 
reduction, 30, 34,185 
regular expression, 36, 311, 478, 478,481 
rehashing, 207 

Reverse Polish notation, 28, see RPN 
right child, 150, 151, 153, 160, 167, 257, 258, 278, 
279, 352 

right subtree, 150-152,154,160,166,254,256,257, 
262,454 
RLE, 108,108 
rolling hash, 110, 208 

root, 150-152, 154, 155, 160, 163, 166, 167, 169, 
216,256-258,262,263,271,279,302, 
352, 392,454, 468, 470,498 
rooted tree, 302, 352, 352 
row constraint, 86, 87 
RPN, 135,135,136 
run-length encoding, see RLE 

scheduling, 370 
sequence, 311,331,332 
alternating, 331 
ascending, 332 
bitonic, 332 
convex, 331 
production, 366 
weakly alternating, 331 
shared memory, 374, 374 
Short Message Service, see SMS 
shortest path, 300, 355,370,509 

Dijkstra's algorithm for, 4,370 
shortest path, unweighted case, 355 
shortest paths, 41, 368 
shortest paths, unweighted edges, 366 
signature, 395 

singly linked list, 25,112,112,115-117,119,122, 
123,125,127,128,130 
deletion from, 122 
sinks, 350 
SMS, 385 

social network, 15, 393, 398 
soft constraint, 396 
sorted doubly linked list, 471, 472 
sorting, 31, 33, 35, 36, 62, 179,188, 234, 235, 239, 
241,245,253, 390 
counting sort, 234,246 
heapsort, 234 
in-place, 234 
in-place sort, 234 
merge sort, 178, 234, 282 


quicksort, 26, 62, 234, 282, 305 
radix sort, 235 
stable, 234 
stable sort, 234 
sources, 350 
space complexity, 2 

spanning tree, 352, see also minimum spanning 
tree 

SQL, 18, 335 
square root, 40,195 
stable sort, 201, 234 

stack, 25, 28,131,131,132,135,138,141,146,160, 
164,186, 355, 358 
deletion from, 146 
height of, 154 
stacking constraint, 285 
starvation, 375, 385 
streaming 

algorithm, 40,220 
fashion input, 182 

string, 25, 25, 29, 33, 34, 36, 94-98,100, 101, 108, 
109,135,138,208,209,212,232,279, 
309-311,321,322,327,366,383,391- 
394,398,399,467-469,479,497,498, 
505 

string matching, 25, 36,94, 330 

Boyer-Moore algorithm for, 34 
Rabin-Karp algorithm for, 109 
strongly connected directed graph, 351 
Structured Query Language, see SQL 
sub-grid constraint, 86 

subarray, 2, 34, 62, 65, 78,179,191, 220, 271, 304, 
305, 347,441-443,453,490,491,507 
subsequence, 263, 311 

longest alternating, 331 
longest bitonic, 332 
longest convex, 331 
longest nondecreasing, 330, 331 
longest weakly alternating, 331 
substring, 109,110,218, 392,393,395,478,479 
subtree, 151,154,162,166, 257,258,271,274,275, 
301,471 

Sudoku, 85, 85, 86, 87, 296 
suffix, 479 

synchronization constraint, 377 
tail 

of a deque, 142 
of a linked list, 112,118,120 
of a queue, 215 
TCP, 407 

time complexity, 2,14 
timestamp, 28,180, 405 
topological ordering, 350, 370 
tree, 352, 352 
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AVL, 29 

BFS, 355, 365 

binary, see binary tree 

binary search, see binary search tree 

decision, 404 

diameter, 300 

free, 352 

Huffman, 498,499 
ordered, 352 
red-black, 29, 254 
rooted, 302, 352 
trie, 468 
triomino, 36, 37 

Ul, 21, 374 

undirected graph, 351, 351, 352, 363-366,368 
Uniform Resource Locators, see URL 
uniformly random permutation, 81 
UNIX, 399 

URL, 10, 320, 398, 399,406, 407 
user interface, see UI 

vertex, 35, 59, 350, 350, 351-355, 362, 364, 365, 
368-370,397,398,511,514 
black vertex in DFS, 362 
connected, 351 
gray vertex in DFS, 362 
white vertex in DFS, 362 

weakly alternating sequence, 331 
weakly connected graph, 351 
weighted directed graph, 511 
weighted undirected graph, 368 
width, 44 
work-queue, 386 
World Wide Web, 406 

XML, 402,407 
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